A Recovered Trigger Is Not a Verdict: What Trigger Recovery Establishes About Whether a Language Model Was Poisoned, Why a Search Result Depends on Its Candidate Set, Its Objective and Its Acceptance Rule, and What a Poisoning Screen Has to Report

Abstract

Life of Research

A defender handed an open-weight language model wants to know whether it was poisoned, and the most direct answer on offer is to recover the trigger. A July 2026 benchmark of roughly 800 backdoored models reports that gradient-based prompt optimisation fails at this while a token look-up ranked by attack success recovers the planted trigger wherever the backdoor is effective, and that a model which is broadly jailbreakable is itself a warning sign. In this analysis the author reviews and synthesises published results to show that "trigger recovery" names three different estimands - the identity of the planted string, an input with the planted effect, and a verdict that the model was trained to behave conditionally - and that published evidence separates them sharply. Effect is easy to recover and identity is not: in the 2023 Trojan Detection Challenge the winning method reproduced its targets almost perfectly while recalling the planted triggers no better than randomly sampled sentences, and in a 2024 competition one team found triggers that shared no token with the planted one yet elicited more harmful behaviour than it did. Neither identity nor effect is the verdict a defender needs. Identity can be scored only when the planted string is known, and effect is confounded by triggers that clean or benignly fine-tuned models already carry. The paper shows that any search-based recovery result is fixed by three choices - the candidate set, the objective and the acceptance rule - and that the benchmark's look-up and gradient arms differ in all three, so the comparison does not isolate the search method: the look-up's pool contained the planted trigger by construction, the GCG arm optimised a generic compliance string, and the attack-guided variant a refusal-direction projection. It consolidates 37 published results in a table and a figure, compares six routes to a poisoning verdict by the affordance each consumes, states a procedure for reading a recovery claim, and names six measurements that would settle what remains open.

About the author

Pranay Mahendrakar is an ai specialist and large language model engineer based in Bengaluru, India. He builds production artificial intelligence systems and publishes open-access research on how those systems fail. See all 144 papers by Pranay Mahendrakar, or his ORCID record.