VK and HSE University build a lightweight AI model for real-time noise suppression and dereverberation

Developer Zakhar Kondaurov, working within a project of the HSE University School of Engineering and Mathematics and VK, under the supervision of Ivan Beskrovny, head of the audio technology team at VK Video, has built a lightweight model that handles both noise suppression and dereverberation of an audio signal in real time.

Why this task is harder than it looks

Most research in speech enhancement is limited to noise suppression, even though modern architectures can handle a harder task: joint noise suppression and dereverberation. The reason is an imbalance in the industry: there are noticeably more benchmarks and ready-made models for noise suppression than for other signal distortions, including reverberation, so new solutions often have nothing to be compared against.

• The project's author is Zakhar Kondaurov, a participant of the HSE University School of Engineering and Mathematics
• The work was done jointly with VK under Ivan Beskrovny, head of the audio technology team at VK Video
• Most existing solutions are trained on audio sampled at 16 kHz, not on full-band 48 kHz
• Noise suppression and dereverberation are today most often handled in sequence by two separate neural networks

What a full-band model offers

The 16 kHz limit sharply narrows the frequency range of the processed signal: upper speech harmonics and fine sound details are simply discarded. A model trained on full-band 48 kHz audio retains noticeably more acoustic information, which matters for voice services such as video calls and streams, where the VK Video team works.

Noise suppression and dereverberation are often handled in sequence today: one neural network first removes extraneous noise, then a second deals with the echo and reverberation of the room. Such a chain uses more memory and processing time than a single model that handles both tasks at once. Combining the tasks in one architecture lets devices with limited resources, such as smartphones, laptops and headsets, process speech locally without sending the audio stream to a server, which reduces both the load on infrastructure and the system's response latency.

What this means

The project shows where speech processing is heading: instead of a chain of narrowly specialized neural networks, one compact full-band model that can run directly on the user's device rather than routing audio through a server to get clean sound in a call or stream.