A Steeper Slope Is Not Yet a New Law: What a Claimed Architectural Change to a Scaling Exponent Has to Survive, Why Studies of Looping, Growth, Routing and Optimizers Reach Opposite Verdicts, and the Floor, Recipe and Ladder Controls That Would Decide It

Abstract

Life of Research

A long-standing working rule of language-model scaling holds that architectural changes move the constant of a scaling law and leave its exponent alone, so that a better design buys a fixed factor of compute at every budget rather than a factor that compounds. A September 2026 study reports the opposite for three interventions on looped transformers - a normalise-and-reinject boundary operator, depth growth during training, and their combination - with compute multipliers over a standard transformer that rise across two decades of compute, and a concurrent study reports a steeper frontier for a looped mixture-of-experts model. Another 2026 study finds that recurrence loses to unique depth at matched compute by a margin that does not close, and a third finds that looping pays at a fixed parameter count without showing an advantage once size and data are optimised for compute. In this analysis the author reviews and synthesises published results to show that these verdicts are not yet answers to one question, because a reported exponent is indexed by five choices that the studies make differently: the axis compute is counted on, how the irreducible floor is fitted, whether each arm has its own training recipe, which shape and allocation rule the ladder follows, and how wide the fitted window is. The paper derives that a lower floor and a steeper exponent both produce a multiplier that rises with scale over a finite window, and that fitting every arm against the baseline's floor reads the first as the second by construction. It shows from the anchor study's own ablations that the boundary operator's exponent gain disappears under the baseline's recipe, and that when executed depth is held fixed its gain in the two-pass pair (Untied-2 against Deep Vanilla) becomes a constant, so that the measured exponent belongs to a ladder and a recipe as much as to a block; the corresponding controls for growth are not reported. It finds the same protocol dependence in two precedents, routing and optimizers, where the trend with scale changed between studies, in the routing case from shrinking to widening. The paper consolidates 24 published results in a table, maps eleven designs onto the five indices, states a classification procedure, and specifies the four controls that would separate a new exponent from a new floor.

About the author

Pranay Mahendrakar is an ai specialist and large language model engineer based in Bengaluru, India. He builds production artificial intelligence systems and publishes open-access research on how those systems fail. See all 119 papers by Pranay Mahendrakar, or his ORCID record.