Kronecker Product Attention Compresses the KV-Cache Up to 10x and Matches MLA on Quality

KV-cache memory has long been the main bottleneck in autoregressive generation: it grows linearly with context length, batch size and model depth. A new arXiv preprint proposes Kronecker Product Attention (KPA), which the authors claim compresses this cache by up to 10x without losing quality against MHA, MQA, GQA, MLA and TPA. The comparison covers four model scales and nine downstream benchmarks under identical training conditions.

How KPA works

The core idea: keys and values are not stored in full. Instead, they are decomposed into a sum of Kronecker products of compact matrix factors. The factors are token-dependent, meaning they are computed from the input rather than fixed in the weights. Only these factors stay in the cache, and the full tensors are reconstructed when attention is computed.

Existing approaches make different trade-offs. MQA and GQA save memory by merging heads, and lose expressiveness as a result. MLA compresses the cache through a latent representation, but it needs a separate "decoupled" branch for RoPE positional encoding. TPA factorizes through the outer product of vectors, which imposes a rigid rank-one structure. The authors say KPA strictly generalizes TPA: the vector outer product emerges as a degenerate case of a Kronecker decomposition, while non-degenerate matrix factors can describe richer block patterns.

A separate technical result concerns positions. The authors introduce a structured rotary operator that is consistent with the Kronecker representation. It preserves RoPE's relative-position property but rotates only the compact factors. As a result, no extra positional-encoding branch, as in MLA, is required.

What the paper claims

• KV-cache compression: up to 10x.
• Comparison with MHA, MQA, GQA, MLA and TPA: results on par or better.
• Scale: four model sizes, nine downstream benchmarks.
• Conditions: identical training for all architectures.
• Positional encoding: a RoPE-compatible operator on the compact factors, with no separate branch.
• Tuning: the "Kronecker head split" is set as a hyperparameter.

The last item may be the most practical. The split determines the trade-off between compression ratio and accuracy, and it can be changed at design time without changing the architecture. A team does not need to rewrite the model to move from milder to more aggressive compression. Retraining with a different value is enough.

What engineers should do, and where to be careful

For now this is a v1 preprint with no independent verification. The abstract specifies neither the model sizes, nor the context lengths, nor the decoding speed on real hardware. A memory gain does not always translate into a latency gain: reconstructing keys and values from the factors adds computation, and its cost depends on the kernels and the GPU. The claimed "up to 10x" refers to the best-case setting. Which configuration achieves it, and at what cost in quality, will become clear from the full text and the code, if the authors release it.

Teams building long-context inference or serving large batches should keep KPA on the candidate list alongside MLA. Three things need checking: how much memory it frees at your context lengths, how quality behaves under different splits, and what throughput you get with your attention kernels. For teams that have already trained a model with GQA or MLA, switching requires retraining. It makes more sense to build KPA into the next pre-training cycle than to try to swap the mechanism in finished weights.