| A Quantitative Analysis of Catastrophic Forgetting in Quantized Spiking Neural Networks Assel Kembay, Karina Aguilar, Jason Eshraghian |
Closed-Loop Neuromorphic Deep Brain Stimulation Using Deep Spiking Q-Networks
Binh Nguyen, Edward Mighetto, Dylan Louie, Chunxiu Yu, Jason Eshraghian
See the agenda here.
Abstract: How can “time” be harnessed to boost neural network performance? The brain is a marvel of computation and memory, processing vast amounts of sensory data with an efficiency that puts modern electronics to shame. Reducing the megawatts consumed by hyperscale datacenters to the mere 10 watts the brain requires demands a fundamental shift – leveraging time. We will explore how temporal dynamics enhance neural network efficiency and performance. We will explore the Matrix-Multiply-free language model, where information is distributed across sequences, requiring the model to “learn to forget” in order to utilize limited cache effectively. Ultimately, by embracing temporal strategies, we pave the way toward neuromorphic computing systems that are not only more efficient but also closer to the elegant and sustainable designs found in nature. This exploration marks a step forward in reducing energy demands while advancing the capabilities of artificial intelligence.
Jason Eshraghian and Steven Abreu will be co-presenting a half-day tutorial at IEEE ISCAS 2025 in London, UK.
Press release link.
Link to the recording.
In collaboration with Intel Labs and the University of Groningen, this work deployed the MatMul-free Language Model on neuromorphic hardware (Intel Loihi 2).
See the preprint here.
SpikeGPT was invited for a presentation at the 2025 International Conference on Learning Representations in Singapore.
Abstract: We introduce NeuroSA, a neuromorphic architecture specifically designed to ensure asymptotic convergence to the ground state of an Ising problem using a Fowler-Nordheim quantum mechanical tunneling based threshold-annealing process. The core component of NeuroSA consists of a pair of asynchronous ON-OFF neurons, which effectively map classical simulated annealing dynamics onto a network of integrate-and-fire neurons. The threshold of each ON-OFF neuron pair is adaptively adjusted by an FN annealer and the resulting spiking dynamics replicates the optimal escape mechanism and convergence of SA, particularly at low-temperatures. To validate the effectiveness of our neuromorphic Ising machine, we systematically solved benchmark combinatorial optimization problems such as MAX-CUT and Max Independent Set. Across multiple runs, NeuroSA consistently generates distribution of solutions that are concentrated around the state-of-the-art results (within 99%) or surpass the current state-of-the-art solutions for Max Independent Set benchmarks. Furthermore, NeuroSA is able to achieve these superior distributions without any graph-specific hyperparameter tuning. For practical illustration, we present results from an implementation of NeuroSA on the SpiNNaker2 platform, highlighting the feasibility of mapping our proposed architecture onto a standard neuromorphic accelerator platform.
Link to the IEEE EMBS Distinguished Lecturers.
Prof. Jason Eshraghian delivered an invited talk titled “A Pathway to Large-Scale Neuromorphic Memristive Systems” at the Atoms to Bits Workshop at the University of Manchester, hosted by Prof. Christoforos Moutafis.
See the recording here.
The next tutorial from UCSC’s Brain-Inspired Machine Learning class is by Ethan Mulle and Abhinandan Singh. The link is here.
They show how to train an SNN using the Forward-Forward Algorithm by Geoff Hinton.
More at this link.
The multi-institutional, large-scale project led by Jason Yik (Harvard), Vijay Janapa Reddi (Harvard), and Charlotte Frenkel (TU Delft) has been published in Nature Communications.
Abstract: Neuromorphic computing shows promise for advancing computing efficiency and capabilities of AI applications using brain-inspired principles. However, the neuromorphic research field currently lacks standardized benchmarks, making it difficult to accurately measure technological advancements, compare performance with conventional methods, and identify promising future research directions. This article presents NeuroBench, a benchmark framework for neuromorphic algorithms and systems, which is collaboratively designed from an open community of researchers across industry and academia. NeuroBench introduces a common set of tools and systematic methodology for inclusive benchmark measurement, delivering an objective reference framework for quantifying neuromorphic approaches in both hardware-independent and hardware-dependent settings. For latest project updates, visit the project website (neurobench.ai).
In the realm of large-scale model training, the efficiency bottleneck often stems from the intensive data communication required between GPUs. Drawing inspiration from the brain’s remarkable efficiency, this talk explores neuromorphic computing’s potential to mitigate this bottleneck. As chip designers increasingly turn to advanced packaging technologies and chiplets, the models running on these heterogeneous platforms must evolve accordingly. Spiking neural networks, inspired by the brain’s method of encoding information over time and its utilization of fine-grained sparsity for information transfer, are perfectly poised to extract the benefits (and limitations) imposed in heterogeneous hardware systems. This talk will delve into strategies for integrating spiking neural networks into large-scale models and how neuromorphic computing, alongside the utilization of chiplets, can surpass the current capabilities of GPUs, paving the way for the next generation of AI systems.
“UC Santa Cruz computer scientists built a more climate-friendly AI”. See more at this link.
