OpenMOSS releases MOSS-Audio, an open audio model that beats larger rivals

OpenMOSS has shipped a rare kind of open-source audio release: MOSS-Audio tries to cover the entire sound-understanding stack in a single model. Instead of a set of separate tools for speech transcription, emotion analysis, background noise, music and timestamp-based questions, the team built one architecture that, by the project's own benchmarks, already outperforms every open alternative it was tested against, including systems more than four times its size.

The project was created by OpenMOSS together with MOSI.AI and the Shanghai Innovation Institute. The release includes four versions: MOSS-Audio-4B-Instruct, MOSS-Audio-4B-Thinking, MOSS-Audio-8B-Instruct and MOSS-Audio-8B-Thinking. The Instruct variants are built to execute requests directly and return predictable answer formats, while the Thinking models target more complex analysis with multi-step reasoning. They are based on Qwen3-4B and Qwen3-8B, and the models total roughly 4.6 and 8.6 billion parameters.

The main bet here is not just size but versatility. MOSS-Audio can transcribe speech with word-level and phrase-level timestamps, identify speaker characteristics and emotions, extract context from ambient sounds, analyze musical style and instruments, answer questions about podcasts, interviews and meetings, and draw conclusions from long audio clips. For developers, this means a simpler pipeline: one system instead of a chain of ASR, an event classifier, a separate music model and timestamp post-processing.

The architecture has three parts: a custom audio encoder, a modality adapter and a language model. Raw audio is first converted into continuous temporal representations at 12.5 Hz, those features are then projected into the LLM's embedding space, and the model generates a text response. The team stresses that it did not use an off-the-shelf audio frontend, training the encoder from scratch instead to get more robust speech representations, better temporal precision and extensibility to different types of sound.

Two technical ideas stand out. The first is DeepStack Cross-Layer Feature Injection: the model does not rely only on the encoder's top layer, but injects features from intermediate levels into the early layers of the LLM, so it does not lose prosody, rhythm, timbre, brief acoustic events and the structure of the background. The second is a time-aware representation with explicit time tokens placed between audio frames during pretraining. This helps the system understand not only what happened but exactly when it happened, which is critical for timestamp ASR, event localization and timeline questions.

According to the reported results, MOSS-Audio-8B-Thinking averages 71.08 points across four general audio benchmarks: 77.33 on MMAU, 64.92 on MMAU-Pro, 66.53 on MMAR and 75.52 on MMSU. That is higher than Step-Audio-R1 with 33 billion parameters, which scored 70.67, and higher than Qwen3-Omni-30B-A3B-Instruct at 67.91. Even the smaller 4B-Thinking version posted an average of 68.37 and beat many larger instruct-only systems.

In speech captioning, 8B-Instruct leads in 11 of 13 categories with an average score of 3.7252, and in ASR it posts the best overall CER of 11.30. On timestamp tasks, the model delivers an AAS of 35.77 on AISHELL-1 and 131.61 on LibriSpeech, clearly ahead of open baseline alternatives.

The key takeaway: open-source audio is starting to shift from narrow models to general-purpose systems that understand speech, music and the sound environment in a single loop. If the numbers hold up in independent tests, MOSS-Audio could become a building block for voice assistants, call analytics, media archive search and long-audio tools, especially where teams need control over the model and local deployment rather than a closed API.