local-ai

Nvidia's Nemotron 3: The First Open-Source Model Worth Running Locally in 2026

Nvidia open-sources a 30B-parameter reasoning model that runs on consumer GPUs with a million-token context window. Here's what makes it different.

Close-up of electronic circuit board with microprocessors

Nvidia just did something unusual. Instead of selling you chips to run other people’s models, the company open-sourced an entire AI development stack: model weights, training data, and the reinforcement learning environments used to train it. The result is Nemotron 3, and the smallest version runs on hardware you might already own.

What Nvidia Released

The Nemotron 3 family comes in three sizes: Nano (30B parameters), Super (~100B), and Ultra (~500B). Only Nano is available now. Super and Ultra arrive in the first half of 2026.

Here’s what makes Nano interesting for local deployment:

  • 30B total parameters, but only 3.5B active per token thanks to mixture-of-experts architecture
  • 1 million token context window - enough to process entire codebases
  • Runs on 24GB VRAM with 4-bit quantization (RTX 3090, 4090)
  • Commercial license included - you can build products on it

The architecture is a hybrid of Mamba-2 and Transformer layers with sparse expert routing. Think of it as combining the efficiency of state-space models with the precision of attention mechanisms when you actually need them. The model alternates between Mamba layers for efficient sequence processing and occasional attention blocks for complex reasoning.

The Numbers That Matter

Benchmarks are marketing, but a few results stand out:

On AIME25 (advanced mathematics with tool use): 99.2%. That’s not a typo. For context, this benchmark includes competition-level math problems that typically require multiple reasoning steps.

On GPQA Diamond (graduate-level science questions): 75.0%

On LiveCodeBench: 68.3%

For long-context tasks using RULER-100, accuracy stays above 90% at 256k tokens and still hits 86.3% at a full million tokens. Most models collapse well before that.

The SWE-Bench score of 38.8% places it solidly in useful territory for coding assistance - not state-of-the-art, but capable enough for real work.

Running It Locally

Ollama support landed shortly after release. The basic setup:

ollama pull nemotron-3-nano
ollama run nemotron-3-nano

For the API:

curl http://localhost:11434/api/chat \
  -d '{
    "model": "nemotron-3-nano",
    "messages": [{"role": "user", "content": "Your prompt here"}]
  }'

If you want more control or need to squeeze onto smaller VRAM, grab a GGUF quantization from Unsloth or LM Studio. The Q4_K_M quantization is the sweet spot for 24GB cards.

For production deployments, vLLM offers better throughput:

pip install -U "vllm>=0.12.0"

vllm serve nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 \
  --served-model-name nemotron \
  --max-num-seqs 8 \
  --tensor-parallel-size 1 \
  --max-model-len 262144 \
  --trust-remote-code

The Reasoning Toggle

Nemotron 3 can expose its chain-of-thought reasoning or hide it. With reasoning enabled (the default), the model shows its work before answering. This improves accuracy on complex tasks but uses more tokens and time.

For simple tasks where you just want an answer:

tokenized_chat = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    enable_thinking=False,
    add_generation_prompt=True,
    return_tensors="pt"
)

You can also set a “reasoning budget” to cap how many tokens the model spends thinking before it must produce an answer. This lets you trade accuracy for latency.

What Nvidia Actually Open-Sourced

This is where Nemotron 3 diverges from typical open-weight releases. Nvidia published:

  • Model weights under the Nemotron Open Model License (commercial use permitted)
  • 3 trillion tokens of pretraining, post-training, and RL data
  • NeMo Gym: the reinforcement learning environments used to train the model
  • Training recipes via GitHub
  • The Nemotron Agentic Safety Dataset for evaluating agent safety

The training data release is significant. You can now see exactly what the model learned from, and use those environments to fine-tune for your own tasks. The NeMo Gym includes environments for competitive coding and advanced mathematics - the same ones that produced those benchmark scores.

How It Compares

Against DeepSeek-V3, Nemotron 3 trails on pure math (AIME25: 72.5% vs 79.8% for DeepSeek R1). Against Qwen 3, the coding performance is competitive. Against Llama 3.3 70B, Nemotron Nano punches above its active parameter count.

The real comparison point is efficiency. Because only 3.5B parameters activate per token, Nemotron 3 Nano achieves 4x higher throughput than its predecessor while matching or exceeding accuracy. For local deployment, this translates to faster responses and lower memory usage.

One user running the GGUF version on a 12GB RX 6700 XT reported 25 tokens per second with the million-token context active. That’s remarkably efficient for a model of this capability.

The Privacy Angle

Running models locally means your prompts never leave your machine. For anyone handling sensitive code, proprietary data, or regulated information, that’s not a minor benefit - it’s a requirement.

With Nemotron 3, you get a reasoning-capable model that can:

  • Process entire codebases (million-token context)
  • Generate and debug code across 43 languages
  • Call external tools via function calling
  • Run entirely offline

The commercial license means you can deploy this in production without API costs or data-sharing agreements. Your infrastructure, your rules.

Hardware Requirements

Minimum for 4-bit quantization:

  • 24GB VRAM (RTX 3090/4090, or AMD equivalent)
  • Linux recommended

Full precision (BF16):

  • A100 80GB or H100 80GB
  • Or distributed across multiple GPUs via tensor parallelism

Optimal:

  • H100 or B200 for maximum throughput
  • Jetson Thor for edge deployment

If you have a recent gaming GPU with 24GB VRAM, you can run this. That’s the practical takeaway.

What’s Coming

Super (~100B parameters) and Ultra (~500B) arrive in H1 2026. These will introduce:

  • Latent MoE: Experts operate on shared representations before projection, enabling 4x more experts at the same inference cost
  • Multi-Token Prediction: Simultaneously predicts multiple future tokens, improving both training accuracy and inference speed through speculative decoding
  • NVFP4 Training: 4-bit floating-point format across the full 25 trillion token pretraining run

The roadmap suggests Nvidia is serious about competing in open-source AI, not just selling hardware to run other vendors’ models.

The Bottom Line

Nemotron 3 Nano is the first 2026 release that makes a compelling case for local deployment. A million-token context, tool calling, exposed reasoning traces, and hardware requirements that match consumer GPUs. The full open-source stack - including training data and RL environments - makes it possible to understand and extend what Nvidia built.

Whether this changes Nvidia’s market position relative to Llama, Qwen, and DeepSeek depends on whether Super and Ultra deliver on their promises. But Nano is shipping now, it runs on hardware people own, and it’s genuinely useful. That’s more than most open-source releases can claim.