Best Local LLM for a Home Assistant Voice Assistant

Which tool-calling local models to point Home Assistant's Ollama integration at, by GPU tier, plus the local speech pieces and the limits HA documents.

Home Assistant can hand voice commands to a model running on your own machine, but only models that support tool calling can actually switch anything on. The right pick depends almost entirely on how much GPU memory sits next to the Home Assistant box.

Hardware tierModel (Ollama tag)What it can doThe catch
No GPU: Home Assistant Green or Raspberry Pi 4 aloneNo LLM; built-in agent with Speech-to-Phrase and PiperFixed home-control sentences, fully localOpen-ended questions, shopping lists and named timers are out
No GPU: a separate PC with spare RAMlfm2.5:8b (LFM2.5-8B-A1B, 5.2 GB)Tool calls with 1.5B active parameters per tokenWeak at knowledge questions by its own card; licence is not Apache
8 GB GPUlfm2.5:8b (5.2 GB)Same model with about 3 GiB left for contextSame catches
12 GB GPUqwen3.5:9b (about 6.5 GB with vision projector)Tools, optional thinking, Apache 2.0Qwen’s card wants 128K context for thinking; HA defaults to 8k
16 GB GPUministral-3:14b (9.1 GB)Native function calling, Apache 2.0, about 7.5 GiB spareMistral itself advises keeping the tool set small
24 GB GPU and upglm-4.7-flash (19 GB at Q4_K_M)30B-A3B MoE, MIT licenceAbout 6 GiB left on a 24 GB card

What Home Assistant asks of the model

The local route for an LLM is the Ollama integration. It “requires an external Ollama server, which is available for macOS, Linux, and Windows”, and the model is named in the integration’s options; “Models will be automatically downloaded during setup” (Home Assistant Ollama integration).

Three requirements come straight from that page. The first is tool support: “Only models that support Tools may control Home Assistant.” The second is restraint on the house side: “If you want to experiment with local LLMs using Home Assistant, we recommend exposing fewer than 25 entities.” The third is a warning about size: “Note that smaller models are more likely to make mistakes than larger models.” The page also labels the feature: “Controlling Home Assistant with Ollama is experimental.” (Home Assistant Ollama integration).

Tools are switched on per agent. The LLM integration page explains that “A new conversation agent starts with Device control only. In Ollama, it starts with no sets.” Device control reads and controls the entities exposed to voice assistants, while the separate Home Assistant set can reach all of Home Assistant and only works for administrators (Home Assistant LLM integration). For voice, tick Device control.

Every pick below carries the tools tag on its Ollama library page: lfm2.5, qwen3.5, ministral-3, gpt-oss and glm-4.7-flash. The broader picture of which local models handle multi-step tool use is in the local agent models guide, and how each runtime parses tool calls is covered in structured output and tool use.

Context window and the cost of exposing entities

Home Assistant sends the exposed entities to the model as part of each request, so the context window matters more here than in a plain chat. The Ollama integration says “Home Assistant defaults to 8k, which is larger than the default value in Ollama Server (2k)”, and adds: “A larger value will better support larger homes with more entities, and smaller values may lower Ollama server RAM usage.” (Home Assistant Ollama integration).

The best-practices page puts the cost in the same terms. Exposing everything slows the built-in parser, “And if you’re using a LLM-based conversation agent, it will incur a higher cost per request, due to the larger context size. Only expose the bare minimum you know you are going to use with voice assistants.” (Home Assistant best practices). Home Assistant publishes no tokens-per-entity figure.

Two settings reduce how often the model is asked at all. With “Prefer handling commands locally” turned on, the built-in agent tries each command first, and “Only commands that it doesn’t understand go to the AI conversation agent.” (Home Assistant voice control). The Ollama page also has a “Think before responding” option, off by default, which “will cause the AI to take longer to respond, but may result in better responses.” (Home Assistant Ollama integration).

The picks, tier by tier

No GPU, Home Assistant box only. A Green has a 1.8 GHz quad-core ARM processor and 4 GB of LPDDR4X (Home Assistant Green), and the Ollama integration expects a separate server anyway. On this tier the sensible assistant has no LLM: the built-in agent, Speech-to-Phrase and Piper, all of which Home Assistant documents as running on a Green or a Raspberry Pi 4 (details below).

No GPU, a separate PC, or an 8 GB card: lfm2.5:8b. This is the same model the site’s agent guide picks for 8 GB. Liquid AI’s card lists 8.3B total and 1.5B active parameters, a 128,000-token context length, and recommends it for “agentic workflows, tool use, structured outputs, multilingual assistants, and on-device personal-assistant applications” (LFM2.5-8B-A1B). The card also claims, as a vendor statement, that it is the “Fastest in its size class on both CPU and GPU inference”, which is the reason to consider it for a CPU-only server. Ollama lists the 8b tag at 5.2 GB at Q4_K_M with a 125K window (Ollama). That is about 4.84 GiB, so an 8 GB card (8 GiB) keeps about 3.2 GiB for the context cache.

The catch is in the same card: “It is not the best fit for heavy programming or knowledge-intensive question answering without retrieval.” For a device-control assistant that is acceptable; for answering general questions it is a real limit. The licence is the LFM Open License v1.0, not Apache 2.0, so read it before any commercial use (LFM2.5-8B-A1B). If the lighter footprint matters more, qwen3.5:4b is a tools-tagged alternative at 2.6 GB of Q4_K_M weights plus a 676 MB vision projector (Ollama).

12 GB: qwen3.5:9b. The agent guide’s 12 GB pick. Ollama’s tag is 5.6 GB of Q4_K_M weights plus a 922 MB BF16 vision projector, about 6.5 GB or 6.07 GiB together, with a 256K window and the tools and thinking tags (Ollama). On a 12 GB card that leaves close to 6 GiB for context. The model is Apache 2.0 (Qwen3.5-9B); on an 8 GB card it would leave under 2 GiB.

The catch: Qwen’s card says “we advise maintaining a context length of at least 128K tokens to preserve thinking capabilities” (Qwen3.5-9B), while Home Assistant’s default is 8k. With “Think before responding” left off, as Home Assistant ships it, the short window matters less.

16 GB: ministral-3:14b. The agent guide’s 16 GB pick, Apriel-1.6-15B, did not appear in the Ollama library when checked on 2026-11-16 (ollama.com/library/apriel returned a 404), and the Home Assistant integration names models the Ollama way. Ministral 3 14B is the substitute: Mistral’s card lists “native function calling and JSON outputting”, a 256k context window and Apache 2.0 (Ministral 3 14B). Ollama’s 14b tag is 9.1 GB at Q4_K_M (Ollama), about 8.48 GiB, leaving about 7.5 GiB on a 16 GB card.

The catch matches Home Assistant’s own advice: Mistral’s card says “Keep the set of tools well-defined and limit their number to the minimum required for the use case”. OpenAI’s gpt-oss:20b is the other tools-tagged option here at 14 GB in MXFP4 with a 128K window (Ollama), but that is about 13.04 GiB of a 16 GiB card, which leaves under 3 GiB for context.

24 GB and up: glm-4.7-flash. The agent guide’s 24 GB pick. Ollama describes it as “a 30B-A3B MoE model”, with the Q4_K_M tag at 19 GB and a 198K window (Ollama); the weights are MIT-licensed (GLM-4.7-Flash). 19 GB is about 17.7 GiB, so a 24 GB card keeps roughly 6 GiB for context.

The local speech pieces

Home Assistant names two local speech-to-text engines and one text-to-speech engine (Home Assistant local voice guide):

  • Speech-to-Phrase “transcribes what it knows”, with “Extremely fast transcription even on a Home Assistant Green or Raspberry Pi 4 (under one second)”. It “Only supports a subset of Assist’s voice commands”.
  • Whisper “will try to transcribe everything”. The guide gives its hardware cost directly: “On a Raspberry Pi 4, it takes around 8 seconds to process incoming voice commands. On an Intel NUC, it is done in under a second.” It recommends Whisper when “You have powerful hardware at home” or when pairing the assistant with an LLM-based agent.
  • Piper is “optimized for the Raspberry Pi 4” and, with medium quality models on a Pi, “it can generate 1.6s of voice in a second”.

The Whisper app is now more than Whisper. Its documentation says that with stt_library on “auto”, English goes to “the parakeet model via the sherpa backend”, Chinese, Cantonese, Japanese and Korean to SenseVoice, Russian to GigaAM, and everything else to faster-whisper. The default model of “auto” “selects tiny-int8 for ARM devices like the Raspberry Pi 4 and base-int8 otherwise” (Whisper app docs).

Piper’s app documentation adds that “On a Raspberry Pi 4, up to the medium models will run with usable speed”, and that its experimental OmniVoice backend “really needs a desktop or server CPU” (Piper app docs). Neither app’s configuration lists a GPU option. Since “Wyoming services can run on another device on your local network” (Wyoming integration), Whisper can run on the same PC as Ollama. The site’s local speech models guide covers other engines, and the Home Assistant setup walkthrough covers installing the pipeline step by step.

Limits Home Assistant documents

  • Small models and tools. “Smaller models may not reliably maintain a conversation when a set of tools is selected”. Home Assistant’s workaround is two Ollama configurations on the same model: one with no AI tools for conversation, one with tools for control (Home Assistant Ollama integration).
  • No sentence triggers. “This integration does not integrate with sentence triggers.” Custom trigger sentences go through the built-in agent (Home Assistant Ollama integration).
  • Scripts need descriptions. Exposed scripts become tools, but “without a good description, the conversation agent doesn’t know the script exists or when to use it” (Home Assistant best practices).
  • Access to the house. The 2026.11 release extended what assistants using the LLM API can do, noted that “an AI conversation can now only be continued by the user who started it”, and advised to “give it the access you are comfortable with and no more” (2026.11 release notes).

The Ollama server should not be reachable from outside the house; the Home Assistant docs only ask that it be “accessible over the network”. The site’s guide to serving a local model to a household covers bind addresses and keeping it on the LAN.

Bottom line

For device control on a Green or a Pi, skip the LLM: Speech-to-Phrase, the built-in agent and Piper are the documented fast path. For free-form requests, run an LLM on a separate machine and choose by GPU memory: lfm2.5:8b up to 8 GB, qwen3.5:9b at 12 GB, ministral-3:14b at 16 GB and glm-4.7-flash at 24 GB. Whichever model you choose, turn on “Prefer handling commands locally”, expose fewer than 25 entities as Home Assistant recommends, and treat device control through Ollama as what Home Assistant calls it: experimental.

Other GPUs and tasks: Local AI by VRAM · RSS