AI's Numerical Tricks Don't Work for Scientific Computing

The AI revolution has changed not only how we talk to computers but also how computers calculate at the most basic level: how they represent numbers. In recent years, the industry has produced dozens of new number formats optimized for machine learning workloads. But attempts to use these formats outside AI have exposed a fundamental problem: what works brilliantly for neural networks turns out to be completely unsuitable for scientific computing.

For decades, the computer industry followed a simple rule: every number gets 64 bits, and that is more than enough for everyone. Users bought new machines every few years and got performance gains essentially for free. But that era ended roughly ten years ago. Moore's Law slowed down while the appetite of AI models grew exponentially. Companies started hunting for any way to save compute and energy, and one of the most effective turned out to be reducing the bit width of numbers. If a neural network doesn't need all 64 bits of precision, why waste them? That's how 16-, 8-, and even 2-bit formats emerged, letting companies train and run models faster and more cheaply.

The problem is that the IEEE 754 standard, which defines 64-bit floating-point numbers, was never designed to scale down well. Its architecture is overkill at low bit counts, and simply truncating it strips away important properties. So the AI industry developed specialized formats such as Google's bfloat16 and NVIDIA's FP8, tailored to the distribution of numbers typical of neural networks. In machine learning, values usually cluster within a certain range, and extreme precision at the edges isn't required.

Scientific computing, however, plays by entirely different rules. Computational physics, fluid dynamics, biological modeling, and engineering simulations work with numbers spread across an enormous range — from subatomic scales to cosmic distances. And they need equally high precision for both very large and very small values. That gap between the needs of AI and science became the starting point for the work of Laslo Hunhold, who recently earned his PhD in computer science at the University of Cologne and joined Barcelona-based startup Openchip as an AI accelerator engineer.

Hunhold developed a number format called takum, based on the earlier posit format. Posit distributes number representations unevenly: frequently used values get more bit patterns and therefore higher precision. That works beautifully for AI — posit concentrates representational density around 1, where typical neural network weights sit. But for scientific computing, the approach is disastrous: precision drops sharply as numbers get larger or smaller, and those are exactly the values that matter most when modeling physical processes.

Takum solves this problem elegantly. Hunhold analyzed the real-world value ranges used in scientific computing across all major disciplines and designed the format so that its dynamic range doesn't shrink as the bit count goes down. That means scientists and engineers could potentially switch to more compact number representations, saving energy and compute time without sacrificing the ability to handle extreme values. According to Hunhold, even a ten percent efficiency gain in a number format translates into ten percent savings across all applications — which, at the scale of global computing capacity, adds up to enormous energy savings.

The significance of this work goes well beyond an academic exercise. As supercomputers and research clusters consume more and more electricity, optimizing how numbers are represented is becoming one of the few remaining levers for improving efficiency without adding more hardware. Notably, Hunhold points out that dozens of new number formats have been proposed in recent years, yet takum remains the only one deliberately designed for scientific computing. Every other innovation in this space serves the machine learning industry exclusively.

The story of takum is a reminder that the AI boom, for all its transformative power, shouldn't overshadow the needs of the rest of computational science. Neural networks aren't the only programs that need efficiency. Physicists modeling the climate, engineers designing bridges, and biologists simulating protein folding deserve the same kind of innovation in basic arithmetic. And if takum gains wide adoption, it could become the invisible foundation that makes the next decade of scientific computing faster, cheaper, and greener.