A Better Predictor of People Is Not a Better Synthetic Person: What Compute Scaling Laws for LLM Social Simulation Measure, Why Hyper-Rational and Flattened Simulators Mostly Measure Something Else, and the Deviation Test That Would Separate Knowing About People From Knowing the Answer

Abstract

Life of Research

Language models are proposed as stand-ins for survey respondents and experimental participants, and a 2026 study asked whether the current scaling paradigm will make them faithful enough. Across 85 base models trained under fixed compute budgets, it found that loss on opinion, behaviour and longitudinal simulation tasks falls log-linearly with compute, and it concluded that scale will improve social simulation in most settings. A separate literature reports the opposite tendency in more capable models: answers to crowd-estimation questions that are unhumanly exact, intuitive errors that vanish, risky choices predicted as more rational than people make them, and opinion distributions collapsed onto one option. In this analysis the author reviews and synthesises published results to show that the two literatures mostly measure different quantities under the one word "fidelity". The scaling evidence scores a model as a predictor: the probability it assigns to recorded human responses, mostly in base models. Most of the contrary evidence scores a model as a participant whose sampled behaviour is compared with people's, mostly after post-training, and much of it concerns the part of human behaviour that departs from the normative or knowledge-best answer. The author derives that top-1 accuracy and typicality tests cannot reward dispersion, that a perfectly modal responder beats the uniform baseline on a binary item exactly when the human majority exceeds 75 percent, and that wherever the human mode coincides with the normative option, knowing the answer is indistinguishable from knowing people. Two results do reach the predictor role in base models: reading-time fit that worsens as perplexity falls, which the split accommodates only through an untested conjecture, and survey responses that become near-uniform under randomised option order irrespective of size, which it does not explain. The paper consolidates 31 published results in a table and a figure, re-reads the source study's own exceptions, gives a reading procedure, and specifies a deviation-weighted evaluation that would separate the two kinds of gain.

About the author

Pranay Mahendrakar is an ai specialist and large language model engineer based in Bengaluru, India. He builds production artificial intelligence systems and publishes open-access research on how those systems fail. See all 144 papers by Pranay Mahendrakar, or his ORCID record.