Sberbank Converted GigaChat3 to Diffusion Mode: How the GFusion Project Works

The GigaChat Pretrain team (Sberbank) published in July 2026 a technical breakdown of the GFusion project — a multi-month experiment on converting the autoregressive model GigaChat3-10B-A1.8B-base to diffusion mode without retraining from scratch. As a result, two experimental checkpoints emerged: GFusion-10B-A1.8B-base and GFusion-10B-A1.8B.

How Diffusion LLMs Differ from Traditional Ones

Autoregressive models — GPT, LLaMA, base GigaChat — generate text token by token, strictly left to right. Once a token is written, it cannot be revised: if the model 'misses' at the start, the only way to fix it is by regenerating the entire response. Diffusion language models (dLLM) work differently: they start with a fully noised or masked sequence and iteratively 'reveal' text over several passes — analogous to how diffusion models reconstruct an image from noise.

Theoretical advantages: dLLMs see the entire context simultaneously, including the right part of text not yet written, which potentially improves coherence of long responses. Parallel denoising opens space for inference acceleration. The project authors themselves emphasize: the direction is 'relatively new' and many ideas are only beginning to undergo practical testing at real scales.

How the Team Conducted the Conversion

The project's key bet is resource efficiency. Pretraining a 10-billion-parameter model from scratch requires enormous GPU-time expenditure and many months of infrastructure work. The GigaChat Pretrain team chose a fundamentally different path: take a ready-made checkpoint of an autoregressive model and adapt it to the diffusion paradigm, preserving everything accumulated during pretraining.

GigaChat3-10B-A1.8B-base is a sparse model: 10 billion parameters in total, but only 1.8 billion are active in each pass (MoE-type architecture), which reduces computational load during inference. As a result of the experiment, two checkpoints with the GFusion suffix emerged.

Project parameters:

• Base model: GigaChat3-10B-A1.8B-base (10B parameters, 1.8B active during inference)
• Approach: AR → dLLM conversion without pretraining from scratch
• Results: GFusion-10B-A1.8B-base and GFusion-10B-A1.8B
• Team: GigaChat Pretrain, Sberbank
• Timeline: internship project, several months

The experiment format itself is remarkable: a non-trivial architectural transition was implemented within an internship — this speaks to the accessibility of tools for teams without the resources of large laboratories.

Why This Matters for Russian-Language AI

GigaChat3 is one of the key open models optimized for the Russian language and Sberbank tasks. Conversion to diffusion mode preserves the model's linguistic 'memory': all knowledge, patterns, and linguistic features accumulated during pretraining remain intact. Only the generation mechanism changes — not what the model knows, but how it expresses it.

Training a quality Russian-language model from scratch is significantly harder than adapting an existing one: a shortage of quality training data and high pretraining costs make 'reworking' ready-made checkpoints strategically attractive. If the GFusion approach proves real advantages in quality or speed, it can be scaled to other existing Russian-language models without months of retraining.

What This Means

GFusion demonstrates that converting autoregressive LLMs to diffusion mode is a practically solvable task, accessible even in an internship format. If further experiments confirm the advantages of the approach, it will open a resource-efficient path to modernizing an entire class of language models — without the costs of a full pretraining cycle.