Local AI by VRAM Tier - 8GB | 12GB | 16GB | 24GB | 32GB
Deep dives: Chat | Coding | Translation | Vision | Speech | Agents
AI agents that browse the web, query databases, control smart-home devices, and chain tools together to solve problems - all running on your own GPU. That’s the promise. The reality depends on whether your model can reliably call functions without hallucinating parameters or losing track of a multi-step plan.
Updated August 2026. Refreshed monthly as models and runtimes change.
What changed this month: the picks are unchanged - LFM2.5-8B-A1B at 8GB, Apriel-1.6-15B-Thinker at 16GB, GLM-4.7-Flash at 24GB and above - but the Qwen references at the 24GB tier have been aligned with the chat and coding guides so all three name the same current-generation model, and every citation has been moved onto model cards and lab repositories.
Function calling is the hardest task for small models. A chat model can get away with an imprecise answer. A model calling a function with the wrong argument crashes the workflow. There’s no partial credit.
The Benchmarks That Matter
- BFCL - the Berkeley Function Calling Leaderboard. Tests single- and multi-turn calls, serial and parallel execution. The standard tool-use benchmark
- TAU-bench / TAU2 / τ²-bench - real-world tool-augmented tasks (retail, airline, telecom domains). Tests whether a model completes end-to-end workflows, not just single calls
- IFEval - instruction-following precision, a good proxy for “does it stay in the required output format?”
A grounding note before the tables: the official BFCL page is JavaScript-only and did not render for scraping, so the function-calling numbers here come from the llm-stats aggregate and from model cards. Different rows use different benchmarks (BFCL vs τ²-bench vs TAU2) that are not directly comparable - a higher number in one column does not automatically beat a lower number in another. Scores published by the lab that trained the model are labelled self-reported. Use these figures to shortlist, then test on your own tools.
Every VRAM figure below is the static weight size at the named quantization and excludes the KV cache, which matters more for agents than for chat: long tool responses fill a context window fast.
What Makes a Good Agent Model?
Beyond benchmarks, practical reliability depends on:
- Structured output - does it format JSON function calls correctly, every time?
- Stop-token discipline - does it stop after the call, or keep generating text that breaks parsing?
- Multi-step planning - can it chain 3+ tool calls without losing the goal?
- Error recovery - when a tool returns an error, does it retry with corrected parameters or spiral?
- Determinism - at temperature 0, is the output parseable every run?
Models explicitly trained for tool use (LFM2.5, Apriel, the Qwen agent line, GLM-4.7-Flash) handle points 1-3 far more consistently than general chat models retrofitted with a tool template.
8GB VRAM {#8gb}
GPUs: RTX 4060, RTX 3060 8GB, RTX 3070
Candidates
| Model | Key figure | VRAM (weights) | License | Note |
|---|---|---|---|---|
| LFM2.5-8B-A1B | Tau2 Telecom 88.07, IFEval 91.84, BFCLv4 48.50 | under ~6 GB | LFM Open License v1.0, commercial use capped at $10M revenue | MoE built for tool calls |
| Qwen3.5-9B | strong native tool format | 5.68 GB (Q4_K_M) | Apache 2.0 | Reliable all-rounder |
Winner: LFM2.5-8B-A1B
Liquid AI’s LFM2.5-8B-A1B is a mixture-of-experts model (8.3B total, ~1.5B active) built specifically for on-device agents. It writes Pythonic function calls by default, self-reports 88.07 on Tau2-Bench Telecom and 91.84 on IFEval, and stays under ~6GB. That combination of a tool-first design and a tiny footprint makes it the 8GB agent pick. Its model card also publishes BFCL scores, self-reported like everything else here: 64.36 on BFCL v3 and 48.50 on BFCL v4. Neither could be reconfirmed on the live Berkeley leaderboard, which carries no entry for this model, so treat them as vendor claims rather than leaderboard placements. Read its licence before shipping too: it is not Apache or MIT but Liquid AI’s own LFM Open License v1.0, which licenses commercial use only below $10 million in annual revenue.
Reliable alternative: Qwen3.5-9B. The Qwen tool-call format is well supported across frameworks and parses cleanly, making the 9B a dependable general agent at 5.68GB of Q4_K_M weights when you want one model for chat and tools. That is the same model the chat guide picks at this tier.
The honest take: 8GB agent models handle 2-3 step tool chains reliably. Beyond that they start losing context. Great for smart-home control, single API calls, and basic retrieval; not for long multi-tool research workflows.
For all use cases at this level, see the 8GB VRAM complete guide.
12GB VRAM {#12gb}
GPUs: RTX 3060 12GB, RTX 4070
Winner: Qwen3.5-9B (headroom), or step toward Apriel-1.6-15B
At 12GB, run Qwen3.5-9B with room for large tool responses (database rows, web-page contents), or move up to the reasoning-first Apriel-1.6-15B-Thinker if your agent has to decide which tool to use, not just how to call it. Apriel’s predecessor ran roughly 10-13GB in independent testing, so on 12GB expect a tight fit with limited context; it is more comfortable at 16GB.
For all use cases at this level, see the 12GB VRAM complete guide.
16GB VRAM {#16gb}
GPUs: RTX 4060 Ti 16GB, RTX 5060, Intel Arc A770
Winner: Apriel-1.6-15B-Thinker
Apriel-1.6-15B-Thinker is the standout mid-tier agent model: a 15B dense model under an MIT license, 131K context, self-reporting 63.50 on BFCL v3 and 69 on Tau-Bench Telecom (with solid retail and airline numbers too). The reasoning training shows up as better planning on multi-step tasks, and MIT licensing plus a single-GPU footprint make it easy to deploy anywhere. 16GB gives it the context room its predecessor lacked at 12GB.
Note the split with the chat guide, which picks GPT-OSS 20B at this tier: that is a general-conversation pick, and it lists Apriel as its reasoning alternative. For tool use specifically, Apriel is the one to run.
Keep a fast fallback: Qwen3.5-9B at 5.68GB of Q4_K_M weights for simple, high-frequency automations where Apriel’s deliberation is overkill.
For all use cases at this level, see the 16GB VRAM complete guide.
24GB VRAM {#24gb}
GPUs: RTX 3090, RTX 4090
This is where local agents get serious - long tool-response chains and multi-step workflows.
Candidates
| Model | Key figure | VRAM (weights) | License | Note |
|---|---|---|---|---|
| GLM-4.7-Flash | τ²-bench 79.5, SWE-bench Verified 59.2 (self-reported) | 19 GB (Q4_K_M) | MIT | Strong end-to-end agent |
| Qwen3.6-27B | current-generation dense Qwen | ~17 GB (Q4) | Apache 2.0 | Same pick as the chat and coding guides |
| Qwen3.5-27B | BFCL v4 68.5% | 16.74 GB (Q4_K_M) | Apache 2.0 | Previous generation, the one BFCL measured |
| Qwen3.5-35B-A3B | BFCL v4 67.3% | 22.02 GB (Q4_K_M) | Apache 2.0 | Previous-generation MoE |
Winner: GLM-4.7-Flash
GLM-4.7-Flash is a MIT-licensed 30B-A3B mixture-of-experts model that self-reports 79.5 on τ²-bench and 59.2 on SWE-bench Verified - the strongest end-to-end agent result that fits a 24GB card. Its Q4_K_M build is 19GB on Ollama, which leaves real KV-cache room on a 24GB card. Its MoE design keeps inference fast, which matters for agents: every tool call is a round trip, so a 5-step workflow means 5+ inference passes. Unsloth’s guide covers the quant options.
Reliable function-caller: the Qwen dense line is the pick when the priority is clean, parseable single- and multi-turn calls rather than long autonomous planning. Qwen3.6-27B is the current generation and the model the chat and coding guides also name at 24GB, at around 17GB of Q4 weights. The BFCL v4 aggregate figures that exist are for the previous generation: Qwen3.5-27B at 68.5%, with the Qwen3.5-35B-A3B MoE close behind at 67.3%. No BFCL v4 entry for the 3.6 generation was found in this pass, so the 3.6 recommendation rests on it being the newer release of the same dense line rather than on a measured tool-use score. All of these fit 24GB with context headroom.
For all use cases at this level, see the 24GB VRAM complete guide.
32GB VRAM {#32gb}
GPUs: RTX 5090
Winner: GLM-4.7-Flash at full precision
At 32GB, run GLM-4.7-Flash at Q8_0, 32GB, or at Q4_K_M with generous context and KV-cache headroom. Genuine full precision is the BF16 build at roughly 60GB, which no single consumer card takes - the same self-reported 79.5 τ²-bench capability, now able to hold long tool histories without eviction. For heavier planning, Qwen3.6-27B is a strong dense alternative at around 17GB of Q4 weights, which at this tier means you can run it at a higher quant or with a very large context instead.
For all use cases at this level, see the 32GB VRAM complete guide.
Cross-Tier Summary
Benchmarks differ by row and are not comparable across columns.
| Tier | Best Pick | Benchmark shown | Practical Chain Length |
|---|---|---|---|
| 8GB | LFM2.5-8B-A1B | Tau2 Telecom 88.07 | 2-3 steps |
| 12GB | Qwen3.5-9B / Apriel-1.6-15B | - | 4-5 steps |
| 16GB | Apriel-1.6-15B-Thinker | BFCL v3 63.50 | 5-6 steps |
| 24GB | GLM-4.7-Flash | τ²-bench 79.5 | 8-10 steps |
| 32GB | GLM-4.7-Flash (full precision) | τ²-bench 79.5 | 10+ steps |
Setting Up a Local Agent
With Ollama + MCP (Model Context Protocol)
MCP connects your local model to external data sources and tools. Qwen-Agent has native MCP support and handles tool routing automatically:
pip install qwen-agent
ollama pull qwen3.5:9b
# Configure your MCP servers in the agent config; Qwen-Agent routes the callsWith LM Studio + Function Calling
LM Studio exposes OpenAI-compatible function calling, so it works with any framework expecting the OpenAI API format. Load your model, enable the API server, and point your framework at http://localhost:1234/v1. Check the Ollama library for the exact current model tags.
What Local Agents Can and Can’t Do
Works well:
- Smart-home control (lights, thermostat, media)
- Simple API integrations (weather, notifications, database queries)
- File operations (organize, rename, extract from documents)
- RAG pipelines (search, retrieve, answer)
- 2-3 step workflows on 8GB, 8-10 steps on 24GB+
Struggles with:
- Complex web browsing (multi-step forms, CAPTCHAs)
- Long planning horizons (10+ steps below the 24GB tier)
- Ambiguous tool selection when several tools could work
- Error recovery from unexpected states (timeouts, format changes)
- Reliable parallel tool execution (3+ tools at once)
Not realistic yet:
- Fully autonomous agents running for hours unsupervised
- Research tasks requiring judgment about source reliability
- Multi-agent coordination
The 24GB+ tier with GLM-4.7-Flash is where local agents become genuinely useful for real workflows. Below that, they work as automation assistants for well-defined, simple tasks.
Security Note
Running an agent locally does not make it safe. A local model with shell access can still delete files or expose data. Always:
- Sandbox agents in containers (Docker, Firejail)
- Limit tool permissions to what’s actually needed
- Log all tool calls for review
- Never give agents write access to critical systems without confirmation gates
For a complete walkthrough, see the AI Agent Containment guide.
All eleven local-AI guides, by hardware tier and by task, are indexed at Local AI by VRAM: Which Models Fit Your GPU.