Neuromorphic Computing Group

Brain-Inspired Systems at UC Santa Cruz

Neuromorphic Computing Group

Category Archives: Research

Preprint Update: Training Spiking Neural Networks Using Lessons from Deep Learning

Posted on by

We submitted this extensive (and opinionated) guide to training spiking neural networks to the Proceedings of the IEEE 18 months ago. During this time, the preprint has reached 100+ citations, snnTorch has cracked 80,000 downloads, and it has helped numerous people enter the field of neuromorphic computing… and much of the content that was true 18 months ago has significantly changed.

While we continue to wait for the peer review process to do its thing, I’ve taken the liberty to revamp the preprint to reflect the rapidly changing world of training and using SNNs.

The latest version includes “Practical Notes” with black magic tricks that have helped us improve the performance of SNNs, code-snippets that reduce verbose explanations, and a fresh account of some of the latest going-ons in the neuroscience-inspired deep learning world..

Thank you to Gregor Lenz, Xinxin Wang and Max Ward for working through this >50 page monster.

Preprint link here.

IEEE Transactions on Circuits and Systems Darlington Best Paper Award

Posted on by

The paper titled “How to Build a Memristive Integrate-and-Fire Neuron for Spiking Neuronal Signal Generation” has been awarded the 2023 IEEE Transaction on Circuits and Systems Darlington Best Paper Award. This paper was led by Prof. Sung-Mo Kang, Prof. Jason Eshraghian and Prof. Leon O. Chua in a collaboration spanning UC Santa Cruz, UC Berkeley, University of Michigan, TU Dresden, and Syungkyunkwan University.

The Darlington Best Paper Award annually recognizes one paper that bridges the gap between theory and practice published in the IEEE Transactions on Circuits and Systems and was presented to authors at the 2023 IEEE International Symposium on Circuits and Systems in Monterey, California.

See the announcement here.

The paper is available via IEEE.

New Preprint: PowerGAN: A Machine Learning Approach for Power Side-Channel Attack on Compute-in-Memory Accelerators led by Ph.D. Candidate Ziyu Wang and Prof. Wei D. Lu

Posted on by

Led by Ziyu Wang and Prof. Wei D. Lu (University of Michigan).PowerGAN

Abstract: Analog compute-in-memory (CIM) accelerators are becoming increasingly popular for deep neural network (DNN) inference due to their energy efficiency and in-situ vector-matrix multiplication (VMM) capabilities. However, as the use of DNNs expands, protecting user input privacy has become increasingly important. In this paper, we identify a security vulnerability wherein an adversary can reconstruct the user’s private input data from a power side-channel attack, under proper data acquisition and pre-processing, even without knowledge of the DNN model. We further demonstrate a machine learning-based attack approach using a generative adversarial network (GAN) to enhance the reconstruction. Our results show that the attack methodology is effective in reconstructing user inputs from analog CIM accelerator power leakage, even when at large noise levels and countermeasures are applied. Specifically, we demonstrate the efficacy of our approach on the U-Net for brain tumor detection in magnetic resonance imaging (MRI) medical images, with a noise-level of 20% standard deviation of the maximum power signal value. Our study highlights a significant security vulnerability in analog CIM accelerators and proposes an effective attack methodology using a GAN to breach user privacy.

Preprint: https://arxiv.org/abs/2304.11056

New Preprint: NeuroBench: Advancing Neuromorphic Computing through Collaborative, Fair and Representative Benchmarking

Posted on by

Led by Jason Yik, Vijay Janapa Reddi (Harvard University), and Charlotte Frenkel (TU Delft), the NeuroBench project aims to drive progress in neuromorphic computing by defining benchmarks for neuromorphic algorithms and systems.

Abstract: The field of neuromorphic computing holds great promise in terms of advancing computing efficiency and capabilities by following brain-inspired principles. However, the rich diversity of techniques employed in neuromorphic research has resulted in a lack of clear standards for benchmarking, hindering effective evaluation of the advantages and strengths of neuromorphic methods compared to traditional deep-learning-based methods. This paper presents a collaborative effort, bringing together members from academia and the industry, to define benchmarks for neuromorphic computing: NeuroBench. The goals of NeuroBench are to be a collaborative, fair, and representative benchmark suite developed by the community, for the community. In this paper, we discuss the challenges associated with benchmarking neuromorphic solutions, and outline the key features of NeuroBench. We believe that NeuroBench will be a significant step towards defining standards that can unify the goals of neuromorphic computing and drive its technological progress. Please visit this http URL for the latest updates on the benchmark tasks and metrics.

Preprint: https://arxiv.org/abs/2304.04640

Website: https://neurobench.ai/

New Paper: “Side-channel attack analysis on in-memory computing architectures” led by Ph.D. candidate Ziyu Wang from the Lu Group published in IEEE Trans. Emerging Topics in Computing

Posted on by

“Side-channel attack analysis on in-memory computing architectures” has been published, led by Ziyu Wang and Prof. Wei Lu (Lu Group), along with collaborators Fan-Hsuan Meng and Yongmo Park.

Abstract—In-memory computing (IMC) systems have great potential for accelerating data-intensive tasks such as deep neural networks (DNNs). As DNN models are generally highly proprietary, the neural network architectures become valuable targets for attacks. In IMC systems, since the whole model is mapped on chip and weight memory read can be restricted, the system acts as a ”black box” for customers. However, the localized and stationary weight and data patterns may subject IMC systems to other attacks. In this paper, we propose a side-channel attack methodology on IMC architectures. We show that it is possible to extract model architectural information from power trace measurements without any prior knowledge of the neural network. We first developed a simulation framework that can emulate the dynamic power traces of the IMC macros. We then performed side-channel attacks to extract information such as the stored layer type, layer sequence, output channel/feature size and convolution kernel size from power traces of the IMC macros. Based on the extracted information, full networks can potentially be reconstructed without any knowledge of the neural network. Finally, we discuss potential countermeasures for building IMC systems that offer resistance to these model extraction attack.

SCA

New Preprint: “SpikeGPT: Generative Pre-Trained Language Model with Spiking Neural Networks” led by incoming Ph.D. candidate Ruijie Zhu

Posted on by

We have released SpikeGPT led by Ruijie Zhu and Qihang Zhao, the largest-scale SNN training via backprop to date, and (to the best of our knowledge) the first generative language spike-based model.

spikegpt-architecture

Abstract: As the size of large language models continue to scale, so does the computational resources required to run it. Spiking neural networks (SNNs) have emerged as an energy-efficient approach to deep learning that leverage sparse and event-driven activations to reduce the computational overhead associated with model inference. While they have become competitive with non-spiking models on many computer vision tasks, SNNs have also proven to be more challenging to train. As a result, their performance lags behind modern deep learning, and we are yet to see the effectiveness of SNNs in language generation. In this paper, we successfully implement `SpikeGPT’, a generative language model with pure binary, event-driven spiking activation units. We train the proposed model on three model variants: 45M, 125M and 260M parameters. To the best of our knowledge, this is 4x larger than any functional backprop-trained SNN to date. We achieve this by modifying the transformer block to replace multi-head self attention to reduce quadratic computational complexity to linear with increasing sequence length. Input tokens are instead streamed in sequentially to our attention mechanism (as with typical SNNs). Our preliminary experiments show that SpikeGPT remains competitive with non-spiking models on tested benchmarks, while maintaining 5x less energy consumption when processed on neuromorphic hardware that can leverage sparse, event-driven activations. Our code implementation is available at https://github.com/ridgerchu/SpikeGPT.

Preprint: https://arxiv.org/abs/2302.13939

Code: https://github.com/ridgerchu/SpikeGPT

New Paper: “BPLC + NOSO: Backpropagation of errors based on latency code with neurons that only spike once at most” led by Ph.D. candidate Seong Min Jin published in Complex Intelligent Systems

Posted on by

BPLC + NOSO has been published, led by Seong Min Jin and Doo Seok Jeong (Jeong Lab), along with collaborators Dohun Kim and Dong Hyung Yoo.

Abstract: For mathematical completeness, we propose an error-backpropagation algorithm based on latency code (BPLC) with spiking neurons conforming to the spike–response model but allowed to spike once at most (NOSOs). BPLC is based on gradients derived without approximation unlike previous temporal code-based error-backpropagation algorithms. The latency code uses the spiking latency (period from the first input spike to spiking) as a measure of neuronal activity. To support the latency code, we introduce a minimum-latency pooling layer that passes the spike of the minimum latency only for a given patch. We also introduce a symmetric dual threshold for spiking (i) to avoid the dead neuron issue and (ii) to confine a potential distribution to the range between the symmetric thresholds. Given that the number of spikes (rather than timesteps) is the major cause of inference delay for digital neuromorphic hardware, NOSONets trained using BPLC likely reduce inference delay significantly. To identify the feasibility of BPLC + NOSO, we trained CNN-based NOSONets on Fashion-MNIST and CIFAR-10. The classification accuracy on CIFAR-10 exceeds the state-of-the-art result from an SNN of the same depth and width by approximately 2%. Additionally, the number of spikes for inference is significantly reduced (by approximately one order of magnitude), highlighting a significant reduction in inference delay.

Paper: https://link.springer.com/article/10.1007/s40747-023-00983-y

Code: https://github.com/dooseokjeong/BPLC-NOSO

New Paper: “Neuromorphic Deep Spiking Neural Networks for Seizure Detection” led by Ph.D. Candidate Yikai Yang Published in Neuromorphic Computing and Engineering

Posted on by

Abstract: The vast majority of studies that process and analyze neural signals are conducted on cloud computing resources, which is often necessary for the demanding requirements of Deep Neural Network (DNN) workloads. However, applications such as epileptic seizure detection stand to benefit from edge devices that can securely analyze sensitive medical data in real-time and personalised manner. In this work, we propose a novel neuromorphic computing approach to seizure detection using a surrogate gradient-based deep Spiking Neural Network (SNN), which consists of a novel Spiking ConvLSTM unit (SPCLU). We have trained, validated, and rigorously tested the proposed SNN model across three publicly accessible datasets, including Boston Children’s Hospital–MIT (CHB-MIT) dataset from the U.S., and the Freiburg (FB) and EPILEPSIAE intracranial EEG (iEEG) datasets from Germany. The average leave-one-out cross-validation AUC score for FB, CHB-MIT, and EPILEPSIAE datasets can reach 92.7%, 89.0%, and 81.1%, respectively, while the computational overhead and energy consumption are significantly reduced when compared to alternative state-of-the-art models, showing the potential for building an accurate hardware-friendly, low-power neuromorphic system. This is the first feasibility study using a deep Spiking Neural Network for seizure detection on several reliable public datasets.

Read more here.

 

New Preprint: “OpenSpike: An OpenRAM SNN Accelerator” led by Undergraduate Researcher Farhad Modaresi Accepted for ISCAS 2023 in Monterey, CA

Posted on by

Farhad Modaresi has led the design and tape-out of a fully open-sourced spiking neural network accelerator in the Skywater 130 process. The design is based on memory macros generated using OpenRAM.

Many of the advances in deep learning this past decade can be attributed to the open-source movement where researchers have been able to reproduce and iterate upon open code bases. With the advent of open PDKs (SkyWater), EDA toolchains, and memory compilers (OpenRAM by co-author Matthew Guthaus), we hope to port rapid acceleration in hardware development to the neuromorphic community. 

Check out the preprint here: https://arxiv.org/abs/2302.01015

GitHub repo with RTL you are welcome to steal: https://github.com/sfmth/OpenSpike

OpenSpike Schematic and Layout

New snnTorch Tutorial: Nonlinear Regression with SNNs (Part I)

Posted on by

We have a new a new snnTorch tutorial/notebook on nonlinear regression with SNNs.

This first part will show you how to train a spiking neuron’s membrane potential to follow a given trajectory over time. Future tutorials will introduce spiking LSTM models, recurrent leaky integrator neurons, and do some more fancy regression tasks.

Link to the tutorial here.

Nonlinear Regression with SNNs

Nonlinear Regression with SNNs

New Preprint: “Intelligence Processing Units Accelerate Neuromorphic Learning” led by Research Scientist Vincent Sun

Posted on by

Vincent Sun and I have been working with Graphcore to speed-up spiking neural net training workloads on deep learning accelerators.

We’ve integrated snnTorch with Intelligence Processing Units (Graphcore) that leverage sparsity and MIMD execution by a factor of 5-10x over GPUs.

We have released a preprint that describes what we did, a rigorous analysis of IPUs vs GPUs for gradient-based SNN training across a variety of benchmarks in SNNs across neuron models, architectures, and propose a few additional tricks to speed up training (e.g., population codes). We also stress-tested the IPUs and found a few corner cases where GPU performance is on-par (recurrent SNNs with 1-1 connectivity).

snn-ipu
Preprint: https://lnkd.in/gaTGjjh3

Code: https://lnkd.in/gTQ2SE7q