A NAS is an odd place to look for a language model, and a good one. The box is already running, already on your network, and already paid for. It has the RAM, or it does not, and unlike a desktop it will still be there tomorrow morning.
Running a local model on one comes down to three decisions: run Ollama in a container rather than as a native install, keep the model files on storage you control, and size the model to the memory the box actually has.
TL;DR
- Run it as a container. Ollama publishes a single CPU-only command for this:
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama(Ollama Docker docs). Synology’s front end is called Container Manager (Synology); QNAP’s is Container Station (QNAP). - Keep the models on a volume. Docker calls volumes “persistent data stores for containers, created and managed by Docker” (Docker). Bind-mount a shared folder instead if you want to browse the weights in the NAS file manager.
- Watch the context window, not just the model. Ollama “uses a context window size of 4096 tokens” by default, and a bigger one costs more memory (Ollama FAQ).
- Size the model to the RAM. qwen3:0.6b is 523MB and qwen3:1.7b is 1.4GB on disk (Ollama library). That file size is the floor, not the total.
- Check where it loaded. The
PROCESSORcolumn inollama psreads100% CPUwhen the model landed in system memory (Ollama FAQ). - Know what the port exposes. A native install “binds 127.0.0.1 port 11434 by default” (Ollama FAQ), but the
ollama/ollamaimage setsOLLAMA_HOST=0.0.0.0:11434(Ollama Dockerfile), so-p 11434:11434puts the API, with no authentication, on every network the NAS is on.
Start from the container manager the NAS already has
Both vendors ship a Docker front end. Synology describes Container Manager as a way to “Run, develop, test, and ship virtualized applications directly on your NAS”, with a Project dashboard that “makes it possible to create, operate, and manage multi-container Docker projects using Compose files” and an Overview dashboard that “displays at-a-glance information for containers, CPU, RAM, and network usage” (Synology). QNAP’s Container Station “lets you install and centrally manage Docker containers on your QNAP device” and is “Integrated with the Docker Hub Registry”, so an image can be pulled and run without touching a terminal (QNAP).
Whichever app you use, the container needs four things set, and all four come from Ollama’s own documentation:
| Setting | Value | Where it comes from |
|---|---|---|
| Image | ollama/ollama | Ollama Docker docs |
| Published port | 11434 | Ollama Docker docs |
| Volume | /root/.ollama | Ollama Docker docs |
| Environment | OLLAMA_HOST, OLLAMA_MODELS, OLLAMA_CONTEXT_LENGTH, OLLAMA_KEEP_ALIVE | Ollama FAQ |
If you also want a chat UI, put it in the same Compose project so both containers share a network. Synology’s Project dashboard reads Compose files directly, and both containers then show up in the Overview dashboard alongside the CPU, RAM and network figures.
The container command, and what the GPU variants need
Ollama’s Docker documentation leads with the case that matches a NAS, under the heading “CPU only”:
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollamaThe other paths in that document each depend on host-side GPU tooling, which is the part a NAS is least likely to have. The NVIDIA route requires installing the NVIDIA Container Toolkit and then passing --gpus=all. The AMD route passes --device /dev/kfd --device /dev/dri and uses the rocm tag of the image. Vulkan is already bundled into ollama/ollama and is “enabled by default when the container can access the GPU devices”, with OLLAMA_VULKAN=0 to turn it off.
Once the container is up, Ollama’s docs show how to pull a model and talk to it:
docker exec -it ollama ollama run llama3.2Note that llama3.2 with no tag pulls llama3.2:latest, which the library lists as the 3b build at 2.0GB (Ollama library); on a NAS with little RAM, name a smaller tag such as llama3.2:1b instead.
CPU architecture is worth checking before you start. The ollama/ollama image is published for both linux/amd64 and linux/arm64 (Docker Hub), and Ollama’s Linux instructions publish a separate native package for ARM64 alongside the amd64 one (Ollama Linux docs):
curl -fsSL https://ollama.com/download/ollama-linux-arm64.tar.zst \
| sudo tar x -C /usrThat is the native-install path, which is a poor fit for a NAS: the setup the docs recommend adds a systemd service and a dedicated user. The container path avoids both. If you want logs from a native install anyway, the same page points at journalctl -e -u ollama.
Where the models live
The -v ollama:/root/.ollama in the run command creates a Docker-managed volume. Docker’s documentation is explicit about what that buys you: volumes are “persistent data stores for containers, created and managed by Docker”, and “When you create a volume, it’s stored within a directory on the Docker host” (Docker).
That leaves two reasonable choices, and they trade off differently.
The default volume. Simple, and Docker owns it end to end: “volumes are completely managed by Docker” (Docker). The tradeoff is that the weights are invisible in the NAS file manager and in your normal backup jobs, so you back them up separately or you do not back them up at all.
A bind mount to a shared folder. Docker’s guidance is to use bind mounts “if you need to access files or directories from both containers and the host” (Docker). On a NAS that usually means pointing the volume at a shared folder, at which point the models show up in the NAS file manager and ride along with the rest of your backup schedule.
Either way, the weights live inside the container’s .ollama directory, which is why the volume is mounted at /root/.ollama in the first place. You can confirm it rather than guess:
docker exec -it ollama ls /root/.ollama/modelsIf you would rather move the directory wholesale, Ollama’s FAQ documents the environment variable for it: “If a different directory needs to be used, set the environment variable OLLAMA_MODELS to the chosen directory” (Ollama FAQ). For reference, a native Linux install keeps them at /usr/share/ollama/.ollama/models.
Size the model to the RAM, and mind the context window
Memory goes to two places: the model weights, and the context window. The first is the number that gets quoted, and the second is the one that decides whether the whole thing fits.
On the weights, the Ollama library lists download sizes directly. The small end of qwen3 runs at qwen3:0.6b (523MB, 40K context) and qwen3:1.7b (1.4GB, 40K context), with qwen3:4b at 2.5GB and a 256K context window (Ollama library). The same page lists llama3.2:1b at 1.3GB with a 128K context, and the smaller llama3.2:1b-instruct-q2_K build at 581MB (Ollama library). These are download sizes of quantized model files, and they are the floor for what the box has to hold, not the total.
The context window is where CPU-only setups run out first. Ollama “uses a context window size of 4096 tokens” by default, and the context-length documentation is blunt about the trade: “Setting a larger context length will increase the amount of memory required to run a model” (Ollama FAQ, Ollama context docs). That same page lists Ollama’s defaults as “based on VRAM”, with 4k below 24 GiB, 32k between 24 and 48 GiB, and 256k at 48 GiB and above. That table is keyed to VRAM, so on a CPU-only box the 4k entry is the one that applies.
Two server settings decide whether you stay inside your memory ceiling. OLLAMA_NUM_PARALLEL sets how many requests one model handles at once, and the FAQ states plainly that “Required RAM will scale by OLLAMA_NUM_PARALLEL * OLLAMA_CONTEXT_LENGTH”. OLLAMA_MAX_LOADED_MODELS sets how many models sit resident, and the FAQ gives the default as three times the number of GPUs, or three for CPU inference (Ollama FAQ). The FAQ qualifies that cap with “provided they fit in available memory”, but a NAS shares that memory with its file services, so it is worth setting both explicitly.
Confirm where it actually loaded
The check is one command. ollama ps shows what is resident and where:
NAME ID SIZE PROCESSOR UNTIL
llama3:70b bcfb190ca3a7 42 GB 100% GPU 4 minutes from nowThe PROCESSOR column is the useful one: 100% GPU means the model loaded entirely into the GPU, 100% CPU means it loaded entirely into system memory, and a split value like 48%/52% CPU/GPU means it was loaded partially onto both (Ollama FAQ). The context-length documentation shows the same output with a CONTEXT column added, which is the fastest way to confirm the context you actually got rather than the one you asked for (Ollama context docs).
A model sitting at 100% CPU on a NAS is working as intended. It will be slow, and the honest framing is that this is a private, always-on, low-power box for light work, not a replacement for a workstation with a discrete GPU. For that, the site has a guide to running Ollama in Docker with a GPU.
Reach it from your laptop and your phone
A native install is loopback-only out of the box: “Ollama binds 127.0.0.1 port 11434 by default. Change the bind address with the OLLAMA_HOST environment variable” (Ollama FAQ). The container is different: the ollama/ollama image already sets OLLAMA_HOST=0.0.0.0:11434 (Ollama Dockerfile), so once the port is published, any device on the LAN can talk to http://<nas-ip>:11434 with no extra setting. That also means the API is open to every network the NAS can be reached on, so check what the published port is exposed to before going further.
If you would rather run a chat UI behind it, Ollama’s FAQ gives a minimal Nginx proxy:
server {
listen 80;
server_name example.com; # Replace with your domain or IP
location / {
proxy_pass http://localhost:11434;
proxy_set_header Host localhost:11434;
}
}One thing to know before a browser extension gets involved: “Ollama allows cross-origin requests from 127.0.0.1 and 0.0.0.0 by default”, and extra origins are opted into with OLLAMA_ORIGINS (Ollama FAQ).
Keep the port on the LAN. The FAQ documents ngrok and Cloudflare Tunnel for reaching a server from outside, so if you want either one, put a reverse proxy that handles authentication in front of the API first.
For a box that also has internet access, Ollama’s local-only mode is worth switching on. The FAQ describes it as a way to run “in local only mode by disabling Ollama’s cloud features”, set either as OLLAMA_NO_CLOUD=1 or as disable_ollama_cloud in ~/.ollama/server.json. On whether prompts leave the machine, the FAQ answers “Ollama runs locally. We don’t see your prompts or data when you run locally”, and adds that cloud-hosted models are processed on Ollama’s side (Ollama FAQ).
What this means
A NAS is a good host for a local model for reasons that have nothing to do with speed. It is on all the time, it is already backed up and already on your network, and it is the box you are least likely to have to explain the purchase of. The model’s quality ceiling is set by the CPU and the RAM, but everything about where the prompts go and whether the box is awake at 3am is already settled.
The three decisions that determine whether it works are all made up front: container rather than native install, a volume you understand, and a model sized to the memory ceiling rather than to the disk you happen to have.
The bottom line
Pull ollama/ollama, publish port 11434, mount a volume at /root/.ollama, and start with a model in the few-hundred-megabyte range. Then run ollama ps and read the PROCESSOR column. If it says 100% CPU, that is a CPU-only box behaving normally.
Raise the context window only as far as the memory budget allows, and set OLLAMA_MAX_LOADED_MODELS and OLLAMA_NUM_PARALLEL explicitly rather than leaving the defaults to decide how much RAM you are really using.