On January 12, 2026, DeepSeek quietly published a paper that might matter more than their upcoming V4 model. It describes a technique called Engram that lets AI models store knowledge in cheap system RAM instead of expensive GPU high-bandwidth memory. The paper and code are fully open-source.
This is what happens when you cut off a competitor’s access to advanced chips: they engineer around the problem.
The Hardware Constraint That Matters
The bottleneck for training and running large language models isn’t raw compute power. It’s memory. Specifically, high-bandwidth memory (HBM) — the fast RAM that sits on the same package as GPU chips.
HBM is expensive. There’s a global shortage. And if you’re a Chinese AI company, US export controls make it even harder to acquire. The latest restrictions explicitly target HBM3e memory, the cutting-edge stuff that enables models like GPT-4 and Claude to run efficiently.
DeepSeek’s CEO Liang Wenfeng has publicly stated that access to AI compute is the company’s “single biggest constraint.” In a technical paper accompanying DeepSeek V3.2’s release, the team concluded that China’s best domestic chip — Huawei’s Ascend 910C — performs at only 60% the level of NVIDIA’s H100.
So DeepSeek did what engineers do: they found a different path.
What Engram Actually Does
Traditional large language models store everything in neural network weights. Every fact, relationship, and pattern the model learns gets encoded into billions of parameters that live in GPU memory. When you ask the model a question, it computes through all those weights to retrieve relevant information.
This approach is wasteful. A lot of what models “know” is simple factual recall — who founded a company, what year something happened, the definition of a word. These lookups don’t require sophisticated reasoning. But they still consume the same expensive GPU memory and compute cycles as complex multi-step problems.
Engram separates knowledge retrieval from reasoning.
Here’s how it works: Instead of storing everything in neural weights, Engram maintains a massive lookup table in regular system RAM (DRAM). When the model encounters a query, it hashes the input text — specifically, 2-gram and 3-gram sequences of tokens — and performs an O(1) lookup to retrieve relevant embeddings. This happens in constant time, regardless of table size.
The retrieved information gets fed into the model’s reasoning layers. But the heavy lifting of storing all that factual knowledge happens in cheap, abundant system RAM instead of scarce GPU HBM.
The Technical Details
The Engram architecture has three key innovations:
1. Tokenizer Compression
Different capitalizations, accents, and whitespace variations of the same word get mapped to a single canonical token. “Apple,” “APPLE,” and “apple” all point to the same lookup entry. This reduces vocabulary size by 23% without losing information.
2. Multi-Head Hashing
Hash collisions are inevitable when you’re compressing arbitrary text into table indices. Engram uses K distinct hash functions simultaneously. When retrieving embeddings, the system aggregates across all heads, reducing noise from any single collision event.
3. Context-Aware Gating
Raw retrieval is noisy. The same N-gram might appear in completely different contexts. Engram implements a gating mechanism where the model’s current hidden state decides whether to trust the retrieved memory.
If the memory aligns with what the model is currently processing, the gate opens. If it conflicts with context, the gate suppresses it almost completely. The Transformer’s reasoning layers stay in control.
The Numbers
DeepSeek tested Engram on a 27-billion parameter model. They reduced the number of routing experts from 72 to 55 and reallocated those parameters to a 5.7B-parameter embedding table.
The results:
Knowledge Benchmarks:
- MMLU: +3.0 points
- CMMLU: +4.0 points
- ARC-Challenge: +3.7 points
Reasoning:
- Big-Bench Hard: +5.0 points
Code and Math:
- HumanEval: +3.0 points
- GSM8K: +2.2 points
- MATH: +2.4 points
Long-Context Retrieval:
- Needle-in-a-Haystack: 97% accuracy (vs. 84.2% baseline)
That last number is the most striking. The ability to find a specific piece of information buried in a long document jumped from 84% to 97%. This is exactly the kind of task where memory lookup outperforms pure computation.
The 100 Billion Parameter Trick
Here’s the number that matters for the hardware constraints: DeepSeek demonstrated that you can offload a 100-billion-parameter embedding table to host DRAM with throughput penalties below 3%.
The secret is asynchronous PCIe prefetching. Engram lookups are deterministic — the indices depend only on the input token sequence, which is known before the forward pass begins. The system can prefetch embeddings from system RAM while the GPU computes the preceding layers.
This effectively bypasses the HBM capacity wall. You can store vastly more knowledge without needing proportionally more expensive GPU memory.
Why This Matters for Export Controls
The US semiconductor export controls are designed to slow China’s AI progress by restricting access to advanced chips. The logic: if Chinese labs can’t get enough high-bandwidth memory, they can’t train or run frontier-scale models.
Engram doesn’t eliminate the need for GPUs. But it reduces the memory requirements significantly. A model that would normally need cutting-edge HBM3e chips might run acceptably on older hardware with more system RAM.
Ray Wang at SemiAnalysis notes that China’s ChangXin Memory Technologies remains “several years behind industry leaders” like Samsung, SK Hynix, and Micron on HBM manufacturing. But Engram doesn’t need cutting-edge HBM. It needs regular DRAM, which is commodity hardware that China can produce domestically.
The 75/25 configuration DeepSeek found optimal — 75-80% of sparse parameters to routed experts, 20-25% to Engram memory — suggests a scaling strategy that works within hardware constraints rather than fighting them.
DeepSeek V4 and What’s Next
Engram isn’t a standalone product. It’s an architectural component that DeepSeek is integrating into their V4 model, expected mid-February 2026.
V4 combines three efficiency innovations DeepSeek has published in the past month:
- Manifold-Constrained Hyper-Connections (mHC) - more efficient layer communication
- Engram conditional memory - knowledge in RAM instead of GPU memory
- DeepSeek Sparse Attention - reduced attention computation for long contexts
The combination could deliver a “striking breakthrough” in training efficiency, according to analysts. Reports suggest DeepSeek V4 will run on consumer hardware — dual RTX 4090s or a single RTX 5090 — while competing with models that require datacenter infrastructure.
If those claims hold up, it’s a proof of concept that algorithmic innovation can substitute for raw hardware advantages.
The Uncomfortable Implication
American AI policy assumes that controlling hardware controls capability. Engram demonstrates that this assumption has limits.
As one researcher put it: “Efficiency innovations from China could influence training practices at American labs, not because of superior resources but because of superior algorithms.”
DeepSeek open-sourced everything. The paper, the code, the embedding tables — all available on GitHub. Western labs can adopt these techniques immediately. The flow of innovation is two-way.
Export controls are still meaningful. They impose real costs and delays on Chinese AI development. But they’re not the insurmountable barrier some policymakers imagine. When you restrict one path, motivated engineers find others.
What to Watch
Three things to monitor as Engram and similar techniques mature:
1. V4 Performance Claims If DeepSeek V4 actually runs on consumer hardware while matching frontier model quality, the economics of the entire AI industry shift.
2. Adoption by Other Labs Will Western companies integrate Engram-style memory architectures? The technique is openly published and doesn’t require exotic hardware. There’s no technical reason not to use it.
3. Policy Response Export control advocates will argue for tightening restrictions. But Engram demonstrates that algorithmic innovation can route around hardware constraints. The policy question becomes: how much can you really control through chip restrictions alone?
DeepSeek continues publishing research at a pace that embarrasses companies with 10x the resources. Engram is another example of constraint driving creativity. The most interesting AI innovations of 2026 might not come from the labs with the most GPUs.
They might come from the ones with the least.
Technical Resources: