An Unknown Unknown Is Defined Relative to an Oracle: Four Places External Judgement Enters Blind-Spot Discovery, and Why the Self-Detection Results Do Not Remove It

Abstract

An unknown unknown is standardly defined as an input on which a model makes a high-confidence mistake. The definition is not incidental to the research problem; it fixes its shape. A quantity defined by the conjunction of high confidence and error cannot be recovered from confidence alone, and the published discovery literature has behaved accordingly: across the twenty methods examined here, spanning 2015 to 2026, not one reports performance that is free of an external correctness signal at every stage, and what changes over the decade is only who supplies it - a crowdworker, a label set, a caption corpus, a stronger model, a reward function, the environment. This paper argues that the useful statement is not that self-detection is impossible, which the literature does not establish and which the learnability results do not say, but that unknown-unknown-ness is a relation between a model and an oracle rather than a property of the model. Four distinct places an oracle enters are separated here: guiding the search, supplying supervision to fit a detector, licensing the reported detection numbers, and defining what counts as an error at all. The first two are usually declared. The third and fourth are usually not, which is how a method can be described as requiring no human intervention while its entire reported performance is conditional on a correctness signal it did not generate. The strongest contrary evidence - that internal states carry truthfulness information, that models predict their own behaviour above chance, that entropy at the level of meaning flags confabulations - is stated at full strength and then read for what it licenses: internal signal that is real, partial, and, on the reported evidence, unstable across datasets and uninformative in the high-confidence region where the target lives. Consequences follow for comparison, since two methods with different oracles measure different sets; for the oracle's own error rate, which is not zero; and for what a deployment claim would have to state. Nine studies that would settle the open parts are named. No experiments are reported here.

The literature search, drafting and citation verification for this paper were carried out with AI assistance under the author's direction. Every citation was machine-verified against the arXiv API and Crossref before inclusion, and every quantitative claim was read back against the cited source's own abstract. The author is responsible for the final text and for all claims made in it.

About the author

Pranay Mahendrakar is an ai specialist and large language model engineer based in Bengaluru, India. He builds production artificial intelligence systems and publishes open-access research on how those systems fail. See all 66 papers by Pranay Mahendrakar, or his ORCID record.