Robust Because Coarse, or Robust Because Planned? A Research Agenda for Training-Data Provenance After the Aggregate-Disclosure Turn, in Which Granularity Buys Statistical Power, Prior Randomisation Buys Soundness, and Per-Example Value Has Neither

Abstract

Regulators have chosen coarse disclosure of training data: the template for the EU AI Act's public summary of training content asks for lists of large datasets, a summary of the most relevant crawled domains and a description of how opt-outs were honoured, and does not require a list of individual works. Compensation schemes, opt-out enforcement and licensing markets, by contrast, need a debt from a model to a particular work, and the technical literature reports that per-example attribution is unstable across training runs and can be manipulated. The natural reading is that provenance is robust where it is coarse and gameable where it pays. In this analysis the author reviews and synthesises published results to show that granularity is not the variable that separates the robust claims from the gameable ones. The paper separates four quantities that travel under the word provenance - composition, inclusion, exposure and contribution - and three parties who may be adversarial - the developer, the contributor and the valuator. Aggregation raises statistical power: membership inference that is at chance for single sentences reaches an AUROC of 0.943 on 500-document collections, and influence estimates for groups track actual group effects with rank correlations of 0.8 to 1.0 in convex models. Power is not soundness. A post-hoc test cannot sample its own null hypothesis at any granularity, while a randomised mark inserted before release gives a bounded false-positive rate for a single rightsholder. Coarse units leak: filtering a corpus by current robots.txt opt-outs cut the share of documents matching New York Times text only from 0.354 to 0.267 percent, much of the remainder being copies republished elsewhere. Per-example contribution is the one quantity that is unstable, gameable by contributors, who raised their compensation share to between 185 and 644 percent of its unperturbed value, and gameable by valuators who choose the utility. The paper consolidates the measured values in a table, maps six obligations to the quantity each needs, gives a decision procedure for choosing an instrument, and names seven measurements that would settle what the published record leaves open.

About the author

Pranay Mahendrakar is an ai specialist and large language model engineer based in Bengaluru, India. He builds production artificial intelligence systems and publishes open-access research on how those systems fail. See all 91 papers by Pranay Mahendrakar, or his ORCID record.