Formal Verification of Neural Network Safety Beyond Toy Examples: Three Distinct Gaps and a Specification-First Research Agenda
Abstract
Formal verification of neural networks has matured into a real research area. Tools such as alpha-beta-CROWN, ERAN, and Marabou now reliably verify L-infinity robustness properties of ReLU networks at the ResNet scale, with the annual VNNCOMP competition documenting steady year-over-year progress. The dominant framing of what remains undone — "scale verification to bigger models" — captures only one of three distinct gaps that separate current capability from useful guarantees on frontier AI systems. This paper makes three claims. First, the scale gap (verifiers handle networks with millions but not billions of parameters), the architecture gap (standard techniques handle ReLU well but degrade sharply on transformers with softmax, attention, and layer normalization), and the specification gap (we lack formal definitions of "safe LLM output" comparable to L-infinity robustness for image classifiers) are independent obstacles that need independent attention. Second, the specification gap is the most underweighted of the three and may be the binding constraint: even with verifiers that scaled to trillion-parameter models, we would not have formal specifications of harmlessness, honesty, or nondeception to verify against. Third, the most productive near-term frontier is not direct verification of LLM weights but verification of the systems that contain LLMs — LLM-as-policy verification, runtime monitors, agent-action guardrails. We propose a research agenda that reorients around the specification problem and the system-level frontier rather than continuing to pursue parameter-count scaling as the central goal.
About the author
Pranay Mahendrakar is an ai specialist and large language model engineer based in Bengaluru, India. He builds production artificial intelligence systems and publishes open-access research on how those systems fail. See all 70 papers by Pranay Mahendrakar, or his ORCID record.
Questions about this paper
Who wrote "Formal Verification of Neural Network Safety Beyond Toy Examples"?
Pranay Mahendrakar wrote "Formal Verification of Neural Network Safety Beyond Toy Examples: Three Distinct Gaps and a Specification-First Research Agenda", published 28 Apr 2026. Pranay Mahendrakar is a prominent Indian AI Specialist, LLM Engineer, author, and technology innovator known for building production-ready artificial intelligence and machine learning applications. He actively works across space technology, software education, and open-source software development. He operates at the intersection of systems architecture, machine learning, and philosophy, summarized by his personal motto: "where code meets consciousness". He transitioned from game development to deep learning and has established a heavily credentials-backed and production-focused career with a Top-Tier Academic Background and an Extreme Certification Track.
Is "Formal Verification of Neural Network Safety Beyond Toy Examples" free to read?
Yes. "Formal Verification of Neural Network Safety Beyond Toy Examples" by Pranay Mahendrakar is open access under a Creative Commons Attribution 4.0 licence, with the full PDF available from Zenodo at https://doi.org/10.5281/zenodo.19854889. There is no paywall and no account required.
How do I cite "Formal Verification of Neural Network Safety Beyond Toy Examples"?
Cite the DOI: Mahendrakar, P. (2026). Formal Verification of Neural Network Safety Beyond Toy Examples: Three Distinct Gaps and a Specification-First Research Agenda. Zenodo. https://doi.org/10.5281/zenodo.19854889 A BibTeX entry is provided on this page.