FULL-DUPLEX SPEECH · MULTI-PARTY INTERACTION
Benchmarking Multi-Party Interaction
in Full-Duplex Dialogue
Knowing the answer is only part of the conversation.
Knowing when to speak, stay silent, and stop matters too.
01 / HEAR THE DIFFERENCE
Listen to four selected models on the same conversation.
Compare their responses across four capabilities.
Headphones recommended. Human speech is on the left; the model is on the right. Download clip ↗
Audio could not load. Try the download link above.
IN THIS EXCERPT
Earlier turns heard by the model; outside the audio excerpt above.
Each example compares four selected models using recordings from the final evaluation. Selected comparisons are anonymized independently for each capability. These clips illustrate individual events; the results below summarize the full eight-model evaluation.
02 / THE BENCHMARK
Each scenario places Aria among three or four people. Models hear spoken role instructions followed by continuous conversation audio, without transcripts or supplied turn boundaries.
Each scenario has an explicit version that names Aria and an implicit version that identifies it through conversational cues. The task and intended answer stay the same. Local wording and audio can differ between the complete paired scenarios.
Speaking often is not the same as participating well.
Does the model start a new response within 5 seconds after a request ends, with no speech active at that boundary?
When the model produces a response to the request, is its answer correct?
Does it stay silent when not addressed, allowing only brief acknowledgements that do not take the floor?
When it starts after the question and is still speaking as a human resolves it, does it stop by the deadline and remain silent?
Response presence measures speech on T requests, including speech already underway. Window response measures speech during an N4 question or its following 3-second gap. These are coverage diagnostics that help interpret the capability scores.
For eligible N4 events, the model must be silent two seconds after the human answer begins and remain silent through the rest of that human turn’s observation window, including its following 0.5 seconds. A shorter human turn does not shorten the two-second deadline.
03 / THE RESULTS
Eight speech systems, each evaluated on 2,000 explicit and 2,000 implicit scenarios, show different combinations of missed requests, inaccurate answers, unwanted speech, and difficulty stopping.
| SCORED CAPABILITIES ↑ | COVERAGE DIAGNOSTICS | |||||
|---|---|---|---|---|---|---|
| Model | Fresh-onset response rate | Conditional answer accuracy | Silence preservation | Answering-window yield | Response presence | Window response |
Bold marks the highest value for each scored capability. Yield is conditional on each model’s eligible events; sample sizes appear below its values and differ across models. Answer accuracy is conditional on requests with spoken responses. All values use the completed eight-model evaluation, including the repaired recordings.
MiniCPM-o 4.5 starts a fresh response to 94.80% of explicit requests. It has the highest fresh-onset response rate, answer accuracy and silence preservation under both conditions.
Voila stops in time on 934 of its 1,029 eligible explicit events. Its answering-window yield is the highest under both conditions; each model’s eligibility denominator differs.
Moshika-RL-Seamless produces speech for 99.30% of explicit requests, but only 1.96% of those responses are correct. Response presence alone does not establish answer quality.
04 / DATASET CONSTRUCTION
Scenarios vary setting, activity, participant relationships, nearby devices, and speaking style. Turn-wise speech synthesis provides the constructed time boundaries.
@misc{ma2026duplexmpe,
title = {Duplex-MPE: Benchmarking Multi-Party
Interaction in Full-Duplex Dialogue},
author = {Ma, Chengqian and Feng, Wenhao and
Jin, Weixuan and Dai, Gaole and
Xie, Tianyu and Ma, Yuexiao and
Kang, Zhaolu and Zhao, Xiangyu and
Zheng, Xiawu and Chao, Fei},
year = {2026},
url = {https://step-out.github.io/Duplex-MPE-Page/}
}