Wrong Together Is an Item-Level Quantity: Why Per-Person Reliance Measures Cannot Say Whether AI Assistance Correlates Human Errors, Why the Direct Probes Disagree, and the Reanalysis That Would Settle It

Abstract

Research on reliance on AI advice scores one decision-maker at a time: how often a person accepts correct advice, how often they reject wrong advice, and what that does to their accuracy. Deployment places many decision-makers in front of the same few models, and the theory of algorithmic monoculture warns that their remaining errors may then fall on the same cases, which erodes second opinions, committees and crowds even while each person improves. In this analysis the author reviews and synthesises published results to show that the two literatures cannot currently be joined, and why. Under two stated assumptions, exchangeable decision-makers and errors that are independent given the case, the pairwise covariance of two people's errors equals the variance, across cases, of the probability of erring. That variance splits into a part that per-person reliance rates do identify, set by the AI's error rate and by the gap between a person's error rate when the AI is wrong and when it is right, and parts that they cannot identify, because the standard metrics pool over cases and discard which cases a person got wrong. The direct probes disagree in the way this predicts. In a lung-cancer study where novices followed wrong AI advice about a third of the time, shared errors fell under assistance; in a mammography study where wrong suggestions were planted and most readers erred on those cases, the identifiable part alone puts the error correlation within reader groups at no less than roughly 0.14 to 0.32. Neither side has reported a scale-free, accuracy-adjusted correlation for aided against unaided decisions on the same cases; the nearest quantity, a committee dependence penalty, is not on a correlation scale. The paper gives a decision procedure for when correlation costs anything, and a reanalysis agenda for datasets that already hold the per-case records needed.

About the author

Pranay Mahendrakar is an ai specialist and large language model engineer based in Bengaluru, India. He builds production artificial intelligence systems and publishes open-access research on how those systems fail. See all 95 papers by Pranay Mahendrakar, or his ORCID record.