How to Safely Download and Verify Open-Source LLMs

The downloaded file is not the file you asked for until the registry says it is. An LLM weights file is a multi-gigabyte binary blob, and every step between a registry URL and a running ollama run or llama-cli is an opportunity for a swap, a tampering, or an executable payload. The defenses below are not theoretical. Hugging Face’s own documentation says pickle files “allow more flexibility by giving the ability to users to run python code at de-serialization time,” and Ollama’s HTTP API exposes a verifying sha256 digest event during pulls. Every mainstream registry today publishes enough metadata to verify what you got, but only if you take the steps.

TL;DR

Pin to a real registry and to a specific revision. Hugging Face revision accepts a full commit hash; Ollama pulls check SHA-256 digests at every layer. A latest link is not a pin.
Prefer model.safetensors over pytorch_model.bin. Safetensors is “a simple format for storing tensors safely (as opposed to pickle) and that is still fast (zero-copy)” - the file itself cannot run code when loaded. Pickle can.
Verify the publisher. Read the model card’s base_model, license, datasets, and new_version fields. Prefer repos whose committing user has a GPG-verified commit.
Use the registry’s own downloader. hf_hub_download and snapshot_download for Hugging Face, ollama pull for Ollama. Both run integrity checks under the hood.
Run the first load in isolation. A throwaway user account, a network namespace, or a container with no outbound. The first 30 seconds is when a payload will phone home or read ~/.ssh.

Pick a real registry and pin the version

“Download from the official source” is the most common advice and the least useful. Every model file lives in at least three places - the upstream lab’s own mirror, Hugging Face, and an Ollama rebuild - and a latest link may quietly point at a different file tomorrow. The first move is to pin.

For Hugging Face, the downloader accepts a revision parameter that takes a full commit hash, branch, tag, or PR ref. Per the official download guide, “When using the commit hash, it must be the full-length hash instead of a 7-character commit hash.” A pinned snapshot_download with a full 40-character hash returns the exact bytes that the pinned commit uploaded. The same call against main returns whatever main is today, which may be a different file by the time you read this sentence.

For Ollama, the /api/pull endpoint streams per-layer status that includes a verifying sha256 digest event after each blob downloads. Blobs land in ~/.ollama/models/blobs/sha256-<digest>, so the SHA-256 is part of the file path; a swap that does not match the digest cannot replace the file. The model manifest references layers by their digests, which is why a “same name, different weights” attack fails at the manifest level rather than at the inference level. The /api/push endpoint requires you to specify “the expected SHA256 digest of the file” up front - the server returns 201 only if the digest you sent matches the bytes you uploaded.

For llama.cpp, the upstream README only documents downloads from Hugging Face via the -hf flag, with llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF as the worked example. The README documents no checksum or signing guidance of its own, which means verification has to happen upstream at the Hugging Face repo where the GGUF was uploaded, not at the llama-cli invocation.

Why safetensors, not pickle

Pickle is the default PyTorch serialization format and the canonical “do not unpickle untrusted data” footgun. The Hugging Face security docs are explicit: pickle “allows more flexibility by giving the ability to users to run python code at de-serialization time,” and a malicious file can import a built-in module like exec and run any Python it likes when you torch.load(...). Trail of Bits’ fickling library will generate a working exploit inside a pickle with a one-liner. Hugging Face’s own Pickle Import scan extracts and lists every Python import a .bin file would trigger, with suspicious imports highlighted, but the platform is candid that this is “not 100% foolproof” and is “best-effort.”

Safetensors is the alternative Hugging Face ships and recommends. Its docs describe it as “a new simple format for storing tensors safely (as opposed to pickle) and that is still fast (zero-copy).” The on-disk file is a structured header followed by raw tensor bytes, with no executable opcodes anywhere in the format, so loading cannot trigger code the way a pickle can. Major projects that consume it include transformers, diffusers, llama.cpp, mlx, ComfyUI, AUTOMATIC1111’s stable-diffusion-webui, InvokeAI, and CivitAI, per the safetensors Featured Projects list.

Practical rule: if a repo ships both model.safetensors and pytorch_model.bin, download only the safetensors. The Hugging Face download guide supports this directly with ignore_patterns=["*.bin"] on snapshot_download. Ollama’s GGUF distribution sidesteps the question entirely - GGUF is its own non-executable container format - but the source PyTorch checkpoint on Hugging Face is usually a pickle unless the publisher also published safetensors. The Hugging Face UI shows whether each file has a safetensors badge next to its name; the CLI takes the same filter.

Read the model card before you trust the model

A model card is not branding. The Hugging Face Model Cards spec requires the YAML header to declare license, datasets, base_model, and tags, with the body documenting the model, its intended uses and potential limitations, the training params, which datasets were used, and the evaluation results. That structure is what lets you answer “who trained this, on what, and what am I allowed to do with it” without reading the weights.

Five checks take 60 seconds:

base_model. For a fine-tune or quant, the field points back to the parent. If a “Qwen 70B” quant has no base_model: Qwen/Qwen3-72B, treat it as an unverified redistribution.
license. An SPDX identifier, or a custom name plus a link to a LICENSE file in the repo. other with no link is a flag.
datasets. If the card names datasets, those repos on the Hub have their own provenance. “Trained on the open internet” without a name is a flag.
tags. not-for-all-audiences is a legitimate tag and means the publisher is being explicit, not that you should skip the file.
new_version. If the card declares a new_version field pointing at a sibling repo, follow that link instead of downloading the older weights.

Hugging Face runs every uploaded file through ClamAV at each commit and shows a safe/unsafe badge per file. The Pickle Import scan additionally lists every Python import a .bin file would trigger, with suspicious imports highlighted. A clean badge is a much weaker signal than an unsigned suspicious badge, but the absence of either badge is the strongest “skip this file” signal you will get without running the file yourself.

Identity: GPG-signed commits and the people behind the repo

Authenticating a commit author is separate from authenticating a file. Hugging Face supports GPG-signed commits: if the publisher uploaded a public key to their HF profile and signed the commit, the Hub shows a green “Verified” badge. A “Verified” badge means a key the publisher controls signed bytes the Hub can show you. It does not mean the publisher is trustworthy or that the weights are not themselves malicious, but it does mean the file you are downloading is the file that key uploaded, and a “no signing status” badge is a downgrade of that claim.

The Hugging Face Security overview page lists the platform’s controls: access tokens, 2FA, Git over SSH, SSO, advanced access controls, ClamAV, pickle scanning, and secrets scanning. A repo whose owner has 2FA enabled and a verified GPG key is a stronger provenance chain than a repo with neither, even if both pass the malware scan.

First-run isolation

Every prior step is about confirming the bytes. The next step is what happens when those bytes run. Treat the first load the way you would treat an executable from an unknown source: assume it might read ~/.ssh, exfiltrate environment variables, or open a reverse shell.

Three practical layers, pick one:

A throwaway user account. Create a local user with no SSH keys, no shell history, no ~/.aws, and run the model under that user. If the model tries to read what is not there, nothing leaks.
A network namespace or firewall block. unshare -n ollama run <model> or an outbound-deny iptables rule stops a payload from phoning home. If the model is offline-only, this also tells you so.
A container with no outbound and a read-only root. docker run --network none --read-only --tmpfs /tmp:size=1g -v /tmp/model:/model:ro is enough for inference; the model cannot write to your home directory and cannot reach the network.

None of these protect you from a model that is bad at its job. They protect you from a model whose weights carry a payload.

The Bottom Line

The defenses exist and are documented. Pin to a full commit hash on Hugging Face or a SHA-256-keyed blob on Ollama, prefer safetensors over pickle, read the model card before the model, treat GPG-verified commits as a stronger provenance signal than unsigned uploads, and run the first load under a user that has nothing worth stealing. None of these are exotic; every one of them is built into the registry or your operating system.