Who Spoke Next Is Not Who Should Have: What Zero-Shot Failure on Multi-Party Turn-Taking Benchmarks Measures, Why the Human Reference Sits Beside the Models, and What Would Show That Knowing When to Speak Must Be Trained

Abstract

An assistant that joins a group conversation has to decide, at every pause, whether to take the floor. Two 2026 benchmarks report that language models prompted zero-shot make this decision badly and that fine-tuning repairs it, and both conclude that knowing when to speak is learned rather than emergent. Other 2025-2026 work reports the opposite pattern on neighbouring tasks: untuned models beat human subjects at predicting the next speaker in meetings, and the predictive uncertainty of a model with no task training locates speaking opportunities better than either prompting or fine-tuning. In this analysis the author reviews and synthesises published results to show that the disagreement is about three different quantities that one speak-or-silent score mixes: discrimination (whether a system speaks more often where speaking is appropriate), operating point (how often it speaks at all, which depends on costs and a role the prompt never states), and the target (whose behaviour the label records). Re-weighting the one published human reference to the benchmark's own category mix, the author finds that human annotators would score about 57 percent balanced accuracy on the Friends portion, below the best zero-shot model's 60.5. What fine-tuning visibly adds is a shift in operating point and agreement with the corpus on the one category where humans and corpus disagree; whether it also adds discrimination, the published results do not identify. The paper gives a decision procedure for what a speak-or-silent result licenses and specifies the score-based, cost-stated and panel-labelled measurements that would show whether the capability is missing. The argument covers text-transcript evaluation most firmly; for full-duplex speech models the evidence is thinner and is treated separately.

About the author

Pranay Mahendrakar is an ai specialist and large language model engineer based in Bengaluru, India. He builds production artificial intelligence systems and publishes open-access research on how those systems fail. See all 93 papers by Pranay Mahendrakar, or his ORCID record.