VK and HSE created a lightweight AI model for noise suppression and dereverberation in real time

Zakhar Kondaurov, a developer within the Engineering-Mathematics School at HSE University and VK, working under the guidance of Ivan Beskrovnyi, head of VK Video's audio technologies team, has created a lightweight model that simultaneously solves the tasks of noise suppression and dereverberation of audio signals in real time.

Why this task is more complex than it appears

Most research in speech enhancement is limited to noise suppression alone, although modern architectures allow solving a more complex problem — joint noise suppression and dereverberation. The reason lies in an industry bias: there are significantly more benchmarks and ready-made models for noise suppression than for dealing with other signal distortions, including reverberation, making it difficult to compare new solutions.

• The project author is Zakhar Kondaurov, participant in the Engineering-Mathematics School at HSE University
• Work was conducted jointly with VK under the guidance of Ivan Beskrovnyi, head of VK Video's audio technologies team
• Most existing solutions are trained on audio at 16 kHz sampling rate, not full-band 48 kHz
• Noise suppression and dereverberation are typically solved sequentially using two different neural networks

What a full-band model provides

The 16 kHz limitation significantly narrows the frequency range in the processed signal — upper harmonics of speech and fine sound details are simply discarded. A model trained on full-band audio at 48 kHz preserves considerably more acoustic information, which is important for voice services like video calls and live streams, where VK Video's team operates.

Often, noise suppression and dereverberation are currently solved sequentially: first one neural network removes extraneous noise, then a second one fights echo and room acoustics. Such a pipeline consumes more memory and processing time than a single model that handles both tasks simultaneously. Combining tasks in one architecture allows devices with limited resources — smartphones, laptops, headsets — to process speech locally without sending the audio stream to a server, thus reducing both infrastructure load and system response latency.

What this means

The project demonstrates where the speech processing industry is headed: instead of a chain of narrowly specialized neural networks — one compact full-band model that can run directly on the user's device, rather than sending audio through a server for clean sound in a call or broadcast.