preprint open access

Neuromorphic Computing for On-Device LLM Inference: Why the Three-Layer Integration Gap Matters More Than the Algorithm Layer

Abstract

Life of Research

Spiking neural networks (SNNs) on neuromorphic hardware are routinely proposed as the path to energy-efficient on-device inference for language models. The framing usually treats this as a single technical question. It is not. There are three distinct technical layers — SNN algorithms for language tasks, neuromorphic hardware platforms, and the integration of the two into actual deployed systems — and each layer has a different state of progress, a different set of obstacles, and a different research community. This paper makes three claims. First, conflating the layers obscures where the genuine gaps lie: SNN-LLM algorithm research has accelerated substantially with SpikeGPT, SpikeLLM, Sorbet, and recent spike-driven LLM constructions, and neuromorphic hardware has matured with Loihi 2, IBM NorthPole, SpiNNaker 2, and BrainChip Akida — but actual deployed SNN-LLM systems running on neuromorphic chips at the edge remain almost nonexistent. The integration layer is the bottleneck, not the algorithms. Second, the energy-efficiency case for neuromorphic-LLM deployment must be made against the right baseline — INT4 or sub-4-bit quantized models running on modern mobile NPUs — not against unquantized FP16 GPU baselines. The honest comparison narrows the advantage substantially and changes which workloads are worth pursuing. Third, the most plausible near-term wins are not as drop-in replacements for mobile-NPU LLM inference but in specific niches: always-on low-rate token processing, sensor-fused language tasks where input is already event-based, and ultra-low-power deployment regimes where mobile NPUs do not operate. We propose a research agenda focused on the integration gap and on rigorous baseline comparisons.

Pranay Mahendrakar, AI specialist

About the author

Pranay Mahendrakar is an ai specialist and large language model engineer based in Bengaluru, India. He builds production artificial intelligence systems and publishes open-access research on how those systems fail. See all 161 papers by Pranay Mahendrakar, or his ORCID record.

Questions about this paper

Who wrote "Neuromorphic Computing for On-Device LLM Inference"?

Pranay Mahendrakar wrote "Neuromorphic Computing for On-Device LLM Inference: Why the Three-Layer Integration Gap Matters More Than the Algorithm Layer", published 28 Apr 2026. Pranay Mahendrakar is an Indian AI specialist and LLM engineer based in Bengaluru, India. He is the Managing Director of SonyTech, Nodal Coordinator at IIRS-ISRO, and an instructor at Tutorials Point. His work covers large language models, natural language processing, computer vision and retrieval-augmented generation. He publishes open-access research papers and is the author of three books: Just AI With Pranay, Multiverse of AI and It's Me LLM.

Is "Neuromorphic Computing for On-Device LLM Inference" free to read?

Yes. "Neuromorphic Computing for On-Device LLM Inference" by Pranay Mahendrakar is open access under a Creative Commons Attribution 4.0 licence, with the full PDF available from Zenodo at https://doi.org/10.5281/zenodo.19854579. There is no paywall and no account required.

How do I cite "Neuromorphic Computing for On-Device LLM Inference"?

Cite the DOI: Mahendrakar, P. (2026). Neuromorphic Computing for On-Device LLM Inference: Why the Three-Layer Integration Gap Matters More Than the Algorithm Layer. Zenodo. https://doi.org/10.5281/zenodo.19854579 A BibTeX entry is provided on this page.

Related research by Pranay Mahendrakar

← All papers by Pranay Mahendrakar