DuET cuts piano-to-score transcription error from 6.56% to 3.64%

The problem and the approach

Automatically transcribing piano performances into musical notation is a complex task. A performer varies the timing of each note and shifts the rhythm, yet the resulting score must capture not only the correct pitches but also the bar structure and how notes line up with strong beats. Traditional approaches have focused either on the note level (pitches sounded) or the frame level (beats and downbeats). Researchers Wei Zeng and Ye Wang from the National University of Singapore proposed a fundamentally different solution: a dual-encoder transformer that handles both tasks at once through mutual attention.

DuET's architecture

The model is built on the following components:

Dual encoding and cross-attention: separate encoders process note pitches and the rhythmic frame grid, linked by bidirectional cross-attention to exchange context. An autoregressive decoder alternates between note and frame representations, predicting score attributes sequentially.

Positional encodings: TimeRoPE establishes a unified performance-time reference, while Time-Pitch Rotary Positional Encoding (TPRoPE) explicitly encodes relative time and pitch relationships. These mechanisms let the model more accurately handle the main sources of error — tempo changes and a performer's phrasing.

Joint training: rather than solving MIDI-to-score transcription (PM2S) and beat-and-downbeat tracking separately, the model treats them as a single task, where context from one level helps the other.

Results on the datasets

The team tested DuET on two datasets:

Gangqinpu — the authors' own dataset, with more than 18,000 performance-score pairs spanning several genres and a particular focus on popular piano music. Here, DuET cut the average MUSTER metric error from 6.56% to 3.64% — an improvement of 2.92 percentage points.

ASAP — a classical dataset of academic repertoire. The result here was more modest: error dropped from 11.30% to 11.22%. The difference stems from the complexity of classical repertoire, where interpretive variation is greater.

Ablation studies confirmed that the gains come from frame-level context, temporal positional encodings, and decoder history masking.

Available resources

The authors have released everything needed to use the model:

Interactive demo: https://huggingface.co/spaces/Wayne42/DuET-demo lets you try the model directly in your browser with your own audio recordings.

Source code and weights: https://github.com/wei-zeng98/DuET contains the full implementation and trained models.

Gangqinpu dataset: the authors plan to release the full dataset of more than 18,000 examples for further research.

The work was published on October 15, 2026 and has already drawn a positive response from the community.