Habr AI shows how to build a Linear Layer in C++ and CUDA in its 'From MNIST to Transformer' series

Habr AI has published part three of its 'From MNIST to Transformer' series. It covers the move from ordinary matrices to tensors and shows how to build a basic Linear Layer by hand, without PyTorch. The material takes the reader to a simple neural network for recognizing MNIST and focuses not on library APIs but on how these operations actually execute on the GPU.

What part three covers

The new article continues the path from low-level CUDA code to the architectures on which Transformer and modern LLMs are later built. Instead of ready-made abstractions, the author invites readers to look directly at the computation: how data sits in memory, how the GPU launches operations, and why even a seemingly elementary neural network layer takes a lot of engineering work. It is a good format for those tired of treating ML frameworks as one-line magic.

The main shift in this part is the move from matrices to tensors. For applied machine learning this is a basic topic, because real data is rarely limited to two dimensions. The author shows how a developer's thinking changes when they work not only with tables of numbers but with multidimensional structures, from which batches, layer weights, intermediate representations and other building blocks of a future model are later assembled. This is the level at which preparation begins for understanding more complex blocks such as attention and embeddings.

What gets implemented by hand

The practical part is built around custom C++ and CUDA code. There is no attempt to hide complexity behind a convenient interface; instead, the reader goes from a mathematical formula to a manual implementation. This approach is useful because it ties abstract linear algebra to concrete development steps: data placement, kernel calls, checking dimension shapes, and understanding where exactly errors or performance losses appear. Without this link, it is hard to feel the difference between a teaching example and a real system.

• Implementation of tensor multiplication
• Creation of the first Linear Layer, i.e. a fully connected layer
• Working with memory and data placement on the GPU
• Linking math and code for training on MNIST
• Assembling a simple network for recognizing handwritten digits

"This is the only way to truly understand how LLMs work."

After implementing the basic operations, the article leads to a custom fully connected layer and then to a small network for recognizing handwritten digits from MNIST. This matters: the material does not stop at isolated primitives but shows how to assemble them into a working chain. It also becomes clear that even a simple classifier rests on a fairly deep stack of knowledge, from math to the structure of video memory and the quirks of parallel computing.

Why it matters

The value of such breakdowns is that they dispel the illusion of ease that a Python wrapper often creates. When a developer sees only ready-made library calls, it is hard to judge the cost of each operation and to understand why a model suddenly hits limits in memory, bandwidth or batch size. Working at the level of tensors and layers helps in reading profilers, designing architecture more carefully and choosing trade-offs between speed, accuracy and implementation complexity more deliberately.

The 'From MNIST to Transformer' series is especially useful for those who want not just to run other people's models but to understand the mechanics of modern AI systems. It does not promise a quick entry and warns outright: expect a lot of code, CUDA, math and manual memory work. But this format gives a foundation that will help when optimizing inference, when reading other people's CUDA implementations, and when trying to understand why some architectural decisions run faster than others.

What this means

For the Russian-speaking audience this is a good signal: demand is shifting from superficial tutorials to engineering analysis of the AI stack's internals. The more such material appears, the less LLMs look like a black box, and the easier it is for developers to move from using ready-made models to tuning and optimizing them with understanding. This matters especially now, as demand among practicing engineers for GPU optimization and knowledge of model internals is growing fast.