Qwen 3.5 4B gets a RISC-V processor inside its weights, with DeltaNet as memory

Divyansh Agrawal has released Qwenputer: a regular Qwen 3.5 4B with a working 32-bit RISC-V processor compiled into its weights. The processor uses the model's own Gated DeltaNet layers as memory. On long-integer arithmetic from the NUPA set, the model rose from 48.6% to 100%, exactly matching the same Qwen with an external Python tool. The setup has no external runtime, sandbox or tool calls; everything is computed in forward passes. The code is on GitHub, and the model is on Hugging Face and ModelScope.

DeltaNet layers as addressable RAM

In Qwen3-Next and Qwen 3.5, three out of every four attention layers are built on Gated DeltaNet, a linear RNN. Each head stores a state matrix, and the delta rule updates it. Agrawal noticed that if you set the gate to 1 (nothing is forgotten), the write strength beta to 1 (the old value is replaced entirely) and encode the keys as one-hot, a write overwrites exactly one row, and a query returns that row's exact contents. The result is genuine addressable memory.

This matters for inference cost. A GDN head's state does not grow no matter how long the conversation runs, while the KV-cache of regular attention grows linearly with context length. The link between the delta rule and "fast weights" has been known since the 2021 work by Schlag, Irie and Schmidhuber. What is new here is that the memory is exact and works inside a live model, reliably enough to support an entire processor.

A compiler instead of training

There is no fine-tuning. The processor is described as a circuit of small steps such as "read a register," "add," and "jump if zero," and a compiler turns each step into a piece of a transformer layer. Attention heads do the lookups, feed-forward blocks do the computation, and DeltaNet heads store the registers, memory and program counter. Each instruction produces several tokens: ADDR names the row to write, VAL carries the value, and STEP closes the instruction. Together, these tokens form the execution trace.

The core implements RV32IM with the Zba extension and a small block for big-number arithmetic. It occupies 11 layers ahead of Qwen's own 32 layers, and the original weights remain unchanged: the computer is added in extra dimensions and layers. For reliable use, Agrawal hooked into the tool-call format the model already knows. Qwen sends instructions, the processor runs its own layers and writes the answer itself. A small driver, RvChat, loads the firmware into the GDN state and keeps trace tokens out of Qwen's layers.

Key figures from the project:

• the official riscv-tests pass, and program results match qemu;
• 18 million instructions in free-running mode without a single error;
• about 230 000 output positions checked against a reference machine, with no discrepancies;
• DeepMind div_big: 97.5% for Qwenputer, 95% for Qwen with Python, 72.5% for base Qwen;
• DeepMind mul_big: 97.5% versus 100% with Python and 55% for base Qwen;
• DeepMind mul_div_multiple_longer: 100% versus 97.5% with Python and 90% for base Qwen;
• DeepMind add_or_sub_big: 97.5%, the same as with Python, and 87.5% for base Qwen;
• NUPA, long integers: from 48.6% to 100%;
• ordinary text within noise: KL of 6e-4 against a noise level of 5e-4.

A 512-file file system and the price of the experiment

On top of the memory, the author added an experimental store with put, append, get, list and delete operations. A file name is hashed into one of 1024 directory slots in the DeltaNet state, and the text lives in other heads in chains of eight tokens per row. On a collision, the next slot is checked, so a lookup takes two or three steps.

The store holds up to 512 files, about 53 000 tokens. The state can be saved at the end of a conversation and loaded in the next one. On questions about facts that changed between chats, Qwen answers correctly 90% of the time, versus 30% when old chats are simply pasted into the context, and 83.

3% versus 78.9% for Python memory tools. The code for this part is in the rv-store branch.

The cost is paid in resources. The checkpoint weighs 18.8 GB, roughly twice the size of base Qwen, although most of that volume is zero padding. The model cannot be run with a plain generate() call; it needs RvChat. Each instruction costs several decoded tokens, and the batch executor produces about 3200 processor tokens per second on an L4. A computation that would take an ordinary CPU milliseconds takes seconds.

Where this could be useful in practice

Today, Qwenputer will not replace a Python sandbox on speed. But it removes an entire layer of infrastructure: no tool server, no isolation and no network round trips, just a single 24 GB GPU. For agent systems, this is interesting as a path to deterministic computation without leaving the model. The author plans to run Python on MicroPython inside the weights via a python() tool, and to speed up the processor itself. A second line of work concerns safety. He has written two follow-ups: one on how hybrid models can hide reasoning in RNN state and evade chain-of-thought monitoring, and one on compiled weights as built-in monitors that the model cannot turn off.

To run it: clone the qwenputer repository, run uv sync --extra hf, download divagr1925/Qwen3.5-4B-Qwenputer via the Hugging Face CLI, and launch chipformer.rv.chat with the calc-bridge.elf firmware.