Neuromorphic Computing for On-Device LLM Inference: Why the Three-Layer Integration Gap Matters More Than the Algorithm Layer

Abstract

Spiking neural networks (SNNs) on neuromorphic hardware are routinely proposed as the path to energy-efficient on-device inference for language models. The framing usually treats this as a single technical question. It is not. There are three distinct technical layers — SNN algorithms for language tasks, neuromorphic hardware platforms, and the integration of the two into actual deployed systems — and each layer has a different state of progress, a different set of obstacles, and a different research community. This paper makes three claims. First, conflating the layers obscures where the genuine gaps lie: SNN-LLM algorithm research has accelerated substantially with SpikeGPT, SpikeLLM, Sorbet, and recent spike-driven LLM constructions, and neuromorphic hardware has matured with Loihi 2, IBM NorthPole, SpiNNaker 2, and BrainChip Akida — but actual deployed SNN-LLM systems running on neuromorphic chips at the edge remain almost nonexistent. The integration layer is the bottleneck, not the algorithms. Second, the energy-efficiency case for neuromorphic-LLM deployment must be made against the right baseline — INT4 or sub-4-bit quantized models running on modern mobile NPUs — not against unquantized FP16 GPU baselines. The honest comparison narrows the advantage substantially and changes which workloads are worth pursuing. Third, the most plausible near-term wins are not as drop-in replacements for mobile-NPU LLM inference but in specific niches: always-on low-rate token processing, sensor-fused language tasks where input is already event-based, and ultra-low-power deployment regimes where mobile NPUs do not operate. We propose a research agenda focused on the integration gap and on rigorous baseline comparisons.