GAFU pulls knowledge out of generation in UMMs and lifts VStar-Bench by 20%

A team of researchers has proposed GAFU, a post-training framework for unified multimodal models (UMMs), meaning systems that both parse and draw images within a single architecture. The idea is to directly use what the model "knows" on the generation side for fine-grained visual understanding tasks. Relative to SFT baselines trained on the same data, the maximum relative gain was 20.00% on VStar-Bench, 17.94% on OmniSpatial and 8.62% on VSR-Bench. The result held across different visual tasks and different UMM architectures.

Why generation notices what understanding misses

Fine-grained visual understanding remains a weak spot for vision-language models. They do worse at telling small object attributes apart, or at working out what is to the left of what and what is in front of what, than at describing a scene as a whole. UMMs offer an unusual opening here: they have a generative pathway that, by its nature, has to store a lot of visual information, since otherwise it could not reconstruct an image.

Until now, such models have mostly used generation indirectly: either as an extra training signal (generative supervision) or through explicit image generation while answering. Almost nobody has systematically examined the native representations of the generative side directly. The authors filled that gap with a dedicated analysis and reached three conclusions:

• generative representations react more strongly to fine-grained semantic changes;
• they pick up task-relevant changes that the understanding side's representations miss;
• this advantage works only when the representations have the correct spatial organization.

The third point matters most in practice. It means that simply "plugging in" generative features is not enough: if the spatial structure is broken, the gain disappears.

G-tokens as a bridge between the two branches

GAFU is built on these observations. The framework takes the visual representations of the generative side and compresses them into a compact set of G-tokens that preserve fine visual details. The understanding branch then queries these tokens and extracts what a specific question requires. The authors call this task-conditioned retrieval, that is, looking up information conditioned on the task.

This is post-training, not a rebuild of the model from scratch. The model does not need to generate an image while answering, so there is no latency from an extra pass through the generator. Information that previously stayed inside the generative pathway becomes available for reasoning.

The comparison is set up soundly: GAFU was tested against SFT baselines on identical data. The gain cannot be written off as the model simply seeing more examples.

Key parameters

• Type: post-training framework for UMMs
• Central element: compact G-tokens built from generative representations
• Mechanism: the understanding branch extracts details conditioned on the task
• VStar-Bench: up to +20.00% relative to SFT
• OmniSpatial: up to +17.94%
• VSR-Bench: up to +8.62%
• Validation: different visual understanding tasks and different UMM architectures

What to do with this in practice

For teams building or fine-tuning UMMs, the main takeaway is that the generative part of a model does not have to be a separate "artist." Its representations can become a source of signal for understanding, and within ordinary post-training at that. This is especially interesting where small objects and scene geometry matter: checking details in images, spatial questions, and other tasks where VLMs currently stumble.

There are also limitations to keep in mind. The reported figures are maximum relative gains, not averages across all configurations, and the effect on VSR-Bench is noticeably more modest than on VStar-Bench. In addition, the method relies on the correct spatial organization of generative representations, so architectures where it is broken may gain less.

For the industry, the takeaway goes beyond a single paper. If generation and understanding in UMMs really do complement each other, the case for unified models over a pairing of two specialized systems gets stronger. GAFU shows one concrete way to extract that synergy without making inference more complex.