A local large language model runs on your computer instead of sending every prompt to a hosted chatbot. That can keep sensitive drafts, notes, and code under your control. It also removes per-message fees and can keep working after the model is downloaded and your internet connection is off.
The catch is hardware. A small model may run comfortably on an ordinary laptop but give weaker answers than a leading hosted service. A larger model needs more memory, takes more storage, and may respond slowly without a suitable GPU. Start small, test your actual task, and upgrade only when the results justify it.
1. Check the computer you already have
You do not need a dedicated AI workstation to begin. You do need realistic expectations about memory.
LM Studio’s current system requirements recommend at least 16GB of RAM on supported Macs and Windows PCs. The same page says an 8GB Apple Silicon Mac can work with smaller models and modest context sizes, while Windows users are advised to have at least 4GB of dedicated video memory. LM Studio supports Apple Silicon Macs running macOS 14 or newer, x64 and ARM Windows systems, and x64 or ARM64 Linux systems. Its x64 Windows build requires AVX2, and its Linux documentation specifies Ubuntu 20.04 or newer.
Memory is shared among the model, its working context, the operating system, and your other applications. Close memory-heavy programs before testing. If a model barely loads, shortening the context or choosing a smaller download is more useful than waiting through constant swapping.
Also check free disk space before downloading. Model files can be large, and neither tool can make a model fit merely because the download completed. If you want task-specific suggestions after the basic setup, see our local model guide for 8GB of VRAM.
2. Choose the simpler route for you
Both options run models locally, but they suit different working styles.
Choose Ollama if you are comfortable with a terminal. It offers short commands, a local API, and easy integration with other software. The official Ollama repository provides installers for macOS, Windows, and Linux, plus a Docker image. It also documents a local chat API at http://localhost:11434/api/chat.
Choose LM Studio if you want a desktop interface. Its documentation describes a built-in model search, chat interface, model management, and local API. It supports GGUF models on its desktop platforms and MLX models on Apple Silicon. LM Studio says it can operate entirely offline after the required software and model weights are present.
Do not install both on day one. They solve much of the same beginner problem, and running two model servers can waste memory or make it unclear which program is answering requests.
3. Install and run your first model
Ollama
Download the installer for your operating system from Ollama’s official repository or download page. After installation, open Terminal, PowerShell, or your Linux shell and run:
ollamaThe interactive menu lets you select a model. You can also start one directly. Ollama’s quickstart gives this example:
ollama run gemma4The first run may need to retrieve model files, so allow time for the download. When the chat opens, enter a short test prompt. Type /bye to leave the session.
LM Studio
Open LM Studio and switch to the Discover tab. Search by a model keyword, a user/model identifier, or a full Hugging Face model URL. LM Studio’s model download guide explains that labels such as Q3_K_S and Q_8 identify different quantized versions of the same model. Quantization reduces file size at some cost to quality. The guide recommends choosing a 4-bit option or higher when the computer can handle it.
Download one modest model, open the Chat view, load it, and send a short prompt. Avoid downloading several variants until you know which one runs acceptably. LM Studio lets you change the model storage location from My Models if your main drive is short on space.
4. Test usefulness and privacy
A successful launch only proves that the model runs. Use three tests before trusting the setup:
- Your real task: Ask for the kind of summary, rewrite, explanation, or code help you actually need. A model that handles trivia but fails your work is not a useful fit.
- A verification task: Give it a short passage and ask questions whose answers are present in that passage. Check every answer yourself. Local models can still invent facts.
- An offline test: Quit the tool, disconnect from the network, reopen it, and run the downloaded model. This confirms that your chosen workflow does not depend on a cloud model or unfinished download.
Local inference improves privacy only when the whole workflow stays local. A local model cannot protect text that you paste into a cloud search tool, remote plugin, external API, or synced chat folder. Check integrations before using confidential material. If you later want a browser-based interface, our Ollama and Open WebUI setup explains the extra layer.
Watch system memory while the model runs. If the computer becomes unresponsive, stop the model and choose a smaller or more compressed variant. If answers become confused during a long conversation, begin a new chat or reduce the retained context. More context consumes more memory and is not automatically more accurate.
Quick checklist
- Confirm that your operating system and processor are supported.
- Start with one small, quantized model rather than the largest available option.
- Use Ollama for a terminal-first setup or LM Studio for a graphical setup.
- Test the model on your real task and check its answers.
- Disconnect from the internet once to confirm offline operation.
- Keep cloud plugins and remote APIs away from sensitive prompts.
- Move to a larger model only if quality is inadequate and memory allows it.
A local LLM is not automatically smarter, safer, or completely private. It is a tool you can inspect and control more directly. The best first setup is the smallest one that answers your real questions well enough without making the rest of your computer miserable.