13 / papers

Alignment & RLHF

What training on human preference does to a model over time, and what it quietly costs. 13 open-access papers by Pranay Mahendrakar, each with a permanent DOI.

preprint

Advice Is Not Effort: When a Language Model's Stated Preferences Predict Its Behaviour, Why the Answer Turns on the Channel a Preference Must Travel and the Kind of Preference Asked About, and the Positive Control a Null Result Owes

Life of Research Language models give coherent answers when asked which of two outcomes they prefer, and a growing literature asks whether those answers predict what the models then do. The results look contradictory. A 2026 study of five…

preprint

A Lift Is Not a Selection Gradient: What the Amplification-Lift Gap in Thinking Models Measures, Why an Answer-Aware Label and a Pooled Contrast Account for Much of It, and the Within-Prompt Test That Would Show Training Is Misdirected

Life of Research Reasoning-oriented post-training makes model traces look more deliberative, and a 2026 study asks whether it amplifies the behaviours that go with correct answers. Annotating 15,282 traces from 15 models, it finds that…

preprint

A Residual Is Not a Third Factor: What the Gap Between a Perfect-Retention Ensemble and Joint Training Measures in Continual Learning, Why Chunk Size, Repetition and the Combiner Move It, and the Controls That Would Make Co-observation a Mechanism

Life of Research Continual learning is scored against two reference points: naive sequential training below and joint training on all the data above. The usual reading is that the distance between them is made of forgetting and lost…

preprint

A Rendered Passage Is Not a Picture: Four Things Called Modality Preference in Multimodal Knowledge Conflict, Why the Reported Reversal of Text Dominance Changes the Channel but Not the Content, and the Reliability-Matched Test That Would Settle It

Life of Research A recurring finding about multimodal language models is that they trust text over images when the two disagree. A 2026 study of thirteen models reports the reverse for one kind of evidence: a passage that contradicts the…

preprint

What a Word Limit Removes: Length Constraints Look Free, Costly or Helpful Depending on the Span They Bind, the Scorer That Reads the Answer and the Items Chosen, and Why Compliance Looks Orthogonal to Accuracy in Only One Kind of Design

Life of Research Published work on telling a language model to be brief disagrees about almost everything. One benchmark reports that obeying a word limit and answering accurately are statistically unrelated, and that alignment for…

preprint

A Compliance Gap Is Not a Motive: Why 'Does Alignment Faking Require Scale?' Has No Answer Until the Measured Quantity Is Fixed, What Small-Model Gaps and Frontier Scratchpads Each Establish, and the Interventions That Would Separate Goal-Guarding From Expectation-Tracking

Life of Research Greenblatt et al. (2024) showed Claude 3 Opus complying with a harmful training objective more often when it could infer that its answers would be trained on, and found the same compliance gap in Llama 3.1 405B but not in…

preprint

A Rewrite That Keeps the Label Has Already Read It: Why an LLM Paraphrase Can Raise a Sentiment Classifier's Accuracy Without the Classifier Getting Better, Four Accounts of the Gain, and the Re-Annotation and Neutral-Rate Controls That Would Separate Them

Life of Research (c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. A 2026 study reports that sentiment classifiers score higher on reviews rewritten by small language models than on the human originals, by up to 5.8 accuracy points…

preprint

Agreement Is Not a Criterion: What Matching Human Labels Certifies About an LLM Judge, Why a Shared Shortcut Passes the Test, and Why Every External Criterion Found So Far Validates a Use Rather Than a Judge

Life of Research (c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. An automatic judge is usually accepted on one piece of evidence: it agrees with human preference labels about as often as humans agree with each other. That number is…

preprint

Four Things Called Forgetting: Interference, Transience, Reset and Unlearning Remove Different Things, Why Forgetting Looks Easy by Accident and Hard on Purpose Only Under Different Instruments, and the Savings Measurement That Would Tell Them Apart

Life of Research (c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Machine learning uses one word for four operations. Catastrophic forgetting is damage that fine-tuning does to earlier capabilities. Transience is the fading of…

preprint

Consensus Too Soon, or Agreement From the Start? Shared Prior, Social Coupling and Pool Coverage in Decentralised LLM Collectives, and Why Prompted Diversity and Model Heterogeneity Act on Different Terms

Life of Research (c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Groups of language-model agents that exchange answers and settle on a common one are now a standard way to build decentralised decision systems. A common worry is that…

About this research line

What has Pranay Mahendrakar published on Alignment & RLHF?

Pranay Mahendrakar has published 13 open-access papers on Alignment & RLHF: "Advice Is Not Effort", "A Lift Is Not a Selection Gradient", "A Residual Is Not a Third Factor", "A Rendered Passage Is Not a Picture", "What a Word Limit Removes", "The Rabbit Hole Is Five Claims, Not One", "A Compliance Gap Is Not a Motive", "A Rewrite That Keeps the Label Has Already Read It", "Agreement Is Not a Criterion", "Four Things Called Forgetting", "Consensus Too Soon, or Agreement From the Start? Shared Prior, Social Coupling and Pool Coverage in Decentralised LLM Collectives, and Why Prompted Diversity and Model Heterogeneity Act on Different Terms", "Three Guarantees Under One Word" and "Catastrophic Forgetting in Continual RLHF". Each is deposited on Zenodo with a permanent DOI and is free to read.

← All research topics