Two Kinds of Missing: Underspecified Inputs Versus Unknown Answers, and Why One Abstention Policy Cannot Serve Both

Abstract

A language model that declines to answer is scored the same way whether the reason is that the question has several readings and it picked the wrong one, or that the question has one reading and the model does not know the fact it names. AbstentionBench (Kirichenko et al., 2025) makes the conflation explicit in its own design: it evaluates abstention across 20 datasets grouped into five sources, among them "questions with unknown answers" and "underspecification," scored under a single abstention metric, and reports that reasoning fine-tuning degrades that metric by 24 percent on average across models. Read apart, the two sources look like different problems with different fixes. QuestBench (Li, Kim and Wang, 2025) frames underspecification as a constraint satisfaction problem with one missing variable, and reports that its models excel at naming the missing variable on its math splits but manage only 40 to 50 percent accuracy on its logic and planning splits, with the paper's own analysis attributing the shortfall to a failure to identify the right question rather than to an inability to solve the underlying problem once it is fully specified. Belief-Augmented Generation (Baan et al., 2026) states the difficulty directly: prompted with their own sampled belief state, models by default rarely clarify or abstain at all, and "disentangling when to clarify from when to abstain remains challenging" even inside a system built for exactly that decision. This paper argues that the difficulty is not incidental. Clarifying is a decision about the input: does this prompt admit more than one well-formed reading, and if so which one is meant. Abstaining is a decision about the model: does it possess the fact the single, well-formed reading asks for. The two questions are answered by different evidence -- properties of the prompt against properties of the model's own knowledge -- and a system that reduces both to one scalar confidence threshold will misroute some fraction of each. The paper leads with the underspecification side, where the evidence base is newer and less consolidated; it treats the unknown-answer side only as the second term of the partition and defers its depth to a companion record. It closes by noting a complication the initial framing of this question did not anticipate: at least one system that decouples detection from execution shows well-calibrated, input-appropriate triggering rather than the across-the-board over-clarification a merged metric would predict, which narrows where the practical cost of the conflation actually falls.

The literature search, drafting and citation verification for this paper were carried out with AI assistance under the author's direction. Every citation was machine-verified against its DataCite DOI record before inclusion, with title and author list checked against the record returned. No experiment was run and no number in this paper was measured or recomputed by its author; every figure is quoted from the paper credited with it. Section 2 states the search procedure and a network condition that limited it (the arXiv query API returned HTTP 406 throughout the search; DataCite was used as the verification route instead, per the project's standing workaround for that outage), so the coverage claims later in the paper can be discounted appropriately. The author is responsible for the final text and for all claims made in it.

About the author

Pranay Mahendrakar is an ai specialist and large language model engineer based in Bengaluru, India. He builds production artificial intelligence systems and publishes open-access research on how those systems fail. See all 66 papers by Pranay Mahendrakar, or his ORCID record.