3 / papers
Opening the model up — circuits, features, and what "explaining a behaviour" actually means.
State-space models did not merely "open a door" beyond attention; they walked through it. Mamba, Mamba-2, Samba, Jamba, Granite 4, Hymba, and a growing family of hybrid attention-SSM architectures are now in production language models…
In-context learning (ICL) — the ability of a transformer language model to acquire a new input-output mapping from a handful of demonstrations in its prompt, with no weight updates — remains the most striking and least understood…
This paper introduces a novel theoretical framework for understanding semantic processing in Large Language Models through the lens of sheaf theory and quantum contextuality. We propose that the semantic structure of LLM-generated text can…
