Nemotron 3.5 Bolt Squeezed Into 24 GB: 74B Model Hits 84 Tokens per Second

Researchers have published a paper on arXiv describing Nemotron 3.5 Bolt 74B-A9B 24GB, a hybrid Mamba-Transformer model with image support. Its goal is to deliver the most speed and quality per gigabyte of memory for local agentic workloads. The model has 74B parameters, 9B of which are active when generating each token, and the whole package fits into 24 GB. On RTX Spark it generates 84 tokens per second. By the authors' estimate, that is more than enough speed for long agentic chains. 109 people worked on the paper, and the preprint is registered under number 2610.33039.

From 120B to 24 GB: how Super VL was shrunk

Bolt was not trained from scratch. It was derived from the Nemotron 3.5 Super VL 120B-A12B family, and the compression ran in two steps.

The first step is called Star Elastic compression. It shrinks the architecture itself: total parameters drop from 120B to 74B (down about 38%), and active parameters drop from 12B to 9B (down 25%).

The second step is 2-bit Quantization-Aware Distillation. The weights are compressed to two bits, and distillation restores quality by training the model with quantization already taken into account. A back-of-the-envelope estimate shows the scale: 74B parameters at 2 bits come to roughly 18.5 GB for the weights alone. The rest of the 24 GB is left for other components and working memory. This is a calculation from the stated figures, not a measurement from the paper.

The hybrid Mamba plus Transformer design also helps save memory and boost speed. The model accepts images alongside text, so it can read screenshots of web pages and interfaces.

LiLiCorr and 84 tokens per second

Compression delivered the size but not the speed, so the team trained a separate speculative drafter, LiLiCorr. The drafter proposes several tokens ahead, and the main model verifies them in a single pass. LiLiCorr is released together with the model, and the authors describe it as part of the recipe for hitting the target speed.

The result is 84 tokens per second on RTX Spark. The authors compare this figure with models that have noticeably fewer active parameters. For an agent that calls tools hundreds of times, scrolls through pages and runs terminal commands, sustained speed matters more than peak speed on a short reply.

What we know about Bolt

• Name: Nemotron 3.5 Bolt 74B-A9B 24GB, part of the Nemotron family.
• Architecture: Mamba-Transformer hybrid, vision-language model.
• Parameters: 74B total, 9B active.
• Memory: 24 GB.
• Parent model: Nemotron 3.5 Super VL 120B-A12B.
• Compression: Star Elastic compression.
• Quantization: 2-bit Quantization-Aware Distillation.
• Drafter: LiLiCorr, released together with the model.
• Profile: reasoning, agentic tool calling, web and terminal work.
• Benchmarks: GDPVal v2 (general agentic capabilities), BrowseComp (web browsing), Terminal-Bench 2.1 (terminal).
• Speed: 84 tokens per second on RTX Spark.
• Publication: arXiv cs.CV, submitted 15 October 2026, 109 authors.

Who this is for

According to the authors, Bolt scores higher than leading models with a similar memory footprint and speed. That holds for general agentic tasks, web browsing and terminal work. The comparison is made specifically in the "fits on a single consumer GPU" class, where alternatives are usually either smaller or slower. There are no independent measurements yet; these are the team's own results.

The practical takeaway for developers is simple. If you are building a local agent that opens websites, runs commands and parses screenshots on its own, 24 GB of memory and 84 tokens per second move that setup from experiment to working tool. Data does not leave for the cloud, and the cost per token comes down to electricity. Three things are worth checking: how 2-bit quantization behaves on your tasks, how much the drafter speeds up your specific workloads, and whether the remaining memory is enough for a long context. The recipe itself is also of interest: compressing a large model and then applying distillation during quantization could become a standard path from data-center weights to local ones.