Lossless at What Length? Near-Lossless KV Cache Compression Is a Claim Indexed by Decode Exposure, Scoring Statistic and Budget Accounting, the Decode-Aware Line Has Answered the First Index Mostly on Final-Answer Accuracy, and Divergence From the Full Cache Is Not by Itself the Defect
Abstract
Key-value cache compression is routinely reported as near-lossless at aggressive budgets: eviction, quantisation and merging methods keep a fifth or a tenth of the cache and match the full-cache score. A 2025-2026 line of work argues that the result was manufactured by evaluation, because the benchmarks that established it are dominated by long prompts and short answers. In this analysis the author reviews and synthesises published results to show that the critique is right about what the headline claims omit and too strong about what follows. A losslessness claim carries three indices the headline rarely states: how many tokens are decoded under the compressed state, which statistic scores the output, and how the budget is accounted. On the first, the same eviction method that scores 100 percent on a 7-digit passkey scores 35 percent on a 64-digit one, and sequence-level divergence from the full cache can only accumulate with length. But decode-aware methods evaluated on reasoning traces of thousands of tokens report near-lossless accuracy, so long decoding does not by itself defeat compression. On the second, aggregate accuracy hides instruction-specific eviction, per-sample failures and rationales that no longer support correct answers; yet divergence from the full cache is not the defect either, since the uncompressed reference itself moves under numerical nondeterminism. On the third, one method's "10 percent" and "34 percent" lossless ratios are fixed budgets of 1,536 and 1,024 tokens divided by different mean lengths. The paper states a reporting procedure and the measurements that would settle what remains open.
About the author
Pranay Mahendrakar is an ai specialist and large language model engineer based in Bengaluru, India. He builds production artificial intelligence systems and publishes open-access research on how those systems fail. See all 89 papers by Pranay Mahendrakar, or his ORCID record.
Questions about this paper
Who wrote "Lossless at What Length? Near-Lossless KV Cache Compression Is a Claim Indexed by Decode Exposure, Scoring Statistic and Budget Accounting, the Decode-Aware Line Has Answered the First Index Mostly on Final-Answer Accuracy, and Divergence From the Full Cache Is Not by Itself the Defect"?
Pranay Mahendrakar wrote "Lossless at What Length? Near-Lossless KV Cache Compression Is a Claim Indexed by Decode Exposure, Scoring Statistic and Budget Accounting, the Decode-Aware Line Has Answered the First Index Mostly on Final-Answer Accuracy, and Divergence From the Full Cache Is Not by Itself the Defect", published 13 Oct 2026. Pranay Mahendrakar is an Indian AI specialist and LLM engineer based in Bengaluru, India. He is the Managing Director of SonyTech, Nodal Coordinator at IIRS-ISRO, and an instructor at Tutorials Point. His work covers large language models, natural language processing, computer vision and retrieval-augmented generation. He publishes open-access research papers and is the author of three books: Just AI With Pranay, Multiverse of AI and It's Me LLM.
Is "Lossless at What Length? Near-Lossless KV Cache Compression Is a Claim Indexed by Decode Exposure, Scoring Statistic and Budget Accounting, the Decode-Aware Line Has Answered the First Index Mostly on Final-Answer Accuracy, and Divergence From the Full Cache Is Not by Itself the Defect" free to read?
Yes. "Lossless at What Length? Near-Lossless KV Cache Compression Is a Claim Indexed by Decode Exposure, Scoring Statistic and Budget Accounting, the Decode-Aware Line Has Answered the First Index Mostly on Final-Answer Accuracy, and Divergence From the Full Cache Is Not by Itself the Defect" by Pranay Mahendrakar is open access under a Creative Commons Attribution 4.0 licence, with the full PDF available from Zenodo at https://doi.org/10.5281/zenodo.23343116. There is no paywall and no account required.
How do I cite "Lossless at What Length? Near-Lossless KV Cache Compression Is a Claim Indexed by Decode Exposure, Scoring Statistic and Budget Accounting, the Decode-Aware Line Has Answered the First Index Mostly on Final-Answer Accuracy, and Divergence From the Full Cache Is Not by Itself the Defect"?
Cite the DOI: Mahendrakar, P. (2026). Lossless at What Length? Near-Lossless KV Cache Compression Is a Claim Indexed by Decode Exposure, Scoring Statistic and Budget Accounting, the Decode-Aware Line Has Answered the First Index Mostly on Final-Answer Accuracy, and Divergence From the Full Cache Is Not by Itself the Defect. Zenodo. https://doi.org/10.5281/zenodo.23343116 A BibTeX entry is provided on this page.