Jason Eshraghian is delivering an invited talk at the 2023 Neuro-Inspired Computing Elements “All Aboard the Open-Source Neuromorphic Hype Train” in San Antonio, Texas, USA in April.
The presentation session will give an overview for neuromorphic hardware developed in open-source processes, and highlight the Tiny Neuromorphic Tape-out project that will take place at the Telluride Neuromorphic Cognition and Engineering Workshop.
Jason Eshraghian gave an invited talk at the 2023 Free and Open Source Silicon (FOSSi) Latch-Up Conference “Open Source Brain-Inspired Neuromorphic Software and Hardware” at UC Santa Barbara, CA, USA.
The presentation session will give an overview for how open source tooling has been used to propose and implement neuromorphic solutions and applications. The presentation will highlight the Tiny Neuromorphic Tape-out project that will take place at the Telluride Neuromorphic Cognition and Engineering Workshop.
The recording is available on YouTube.
“Side-channel attack analysis on in-memory computing architectures” has been published, led by Ziyu Wang and Prof. Wei Lu (Lu Group), along with collaborators Fan-Hsuan Meng and Yongmo Park.
Abstract—In-memory computing (IMC) systems have great potential for accelerating data-intensive tasks such as deep neural networks (DNNs). As DNN models are generally highly proprietary, the neural network architectures become valuable targets for attacks. In IMC systems, since the whole model is mapped on chip and weight memory read can be restricted, the system acts as a ”black box” for customers. However, the localized and stationary weight and data patterns may subject IMC systems to other attacks. In this paper, we propose a side-channel attack methodology on IMC architectures. We show that it is possible to extract model architectural information from power trace measurements without any prior knowledge of the neural network. We first developed a simulation framework that can emulate the dynamic power traces of the IMC macros. We then performed side-channel attacks to extract information such as the stored layer type, layer sequence, output channel/feature size and convolution kernel size from power traces of the IMC macros. Based on the extracted information, full networks can potentially be reconstructed without any knowledge of the neural network. Finally, we discuss potential countermeasures for building IMC systems that offer resistance to these model extraction attack.
We have released SpikeGPT led by Ruijie Zhu and Qihang Zhao, the largest-scale SNN training via backprop to date, and (to the best of our knowledge) the first generative language spike-based model.
Abstract: As the size of large language models continue to scale, so does the computational resources required to run it. Spiking neural networks (SNNs) have emerged as an energy-efficient approach to deep learning that leverage sparse and event-driven activations to reduce the computational overhead associated with model inference. While they have become competitive with non-spiking models on many computer vision tasks, SNNs have also proven to be more challenging to train. As a result, their performance lags behind modern deep learning, and we are yet to see the effectiveness of SNNs in language generation. In this paper, we successfully implement `SpikeGPT’, a generative language model with pure binary, event-driven spiking activation units. We train the proposed model on three model variants: 45M, 125M and 260M parameters. To the best of our knowledge, this is 4x larger than any functional backprop-trained SNN to date. We achieve this by modifying the transformer block to replace multi-head self attention to reduce quadratic computational complexity to linear with increasing sequence length. Input tokens are instead streamed in sequentially to our attention mechanism (as with typical SNNs). Our preliminary experiments show that SpikeGPT remains competitive with non-spiking models on tested benchmarks, while maintaining 5x less energy consumption when processed on neuromorphic hardware that can leverage sparse, event-driven activations. Our code implementation is available at https://github.com/ridgerchu/SpikeGPT.
Preprint: https://arxiv.org/abs/2302.13939
Code: https://github.com/ridgerchu/SpikeGPT
BPLC + NOSO has been published, led by Seong Min Jin and Doo Seok Jeong (Jeong Lab), along with collaborators Dohun Kim and Dong Hyung Yoo.
Abstract: For mathematical completeness, we propose an error-backpropagation algorithm based on latency code (BPLC) with spiking neurons conforming to the spike–response model but allowed to spike once at most (NOSOs). BPLC is based on gradients derived without approximation unlike previous temporal code-based error-backpropagation algorithms. The latency code uses the spiking latency (period from the first input spike to spiking) as a measure of neuronal activity. To support the latency code, we introduce a minimum-latency pooling layer that passes the spike of the minimum latency only for a given patch. We also introduce a symmetric dual threshold for spiking (i) to avoid the dead neuron issue and (ii) to confine a potential distribution to the range between the symmetric thresholds. Given that the number of spikes (rather than timesteps) is the major cause of inference delay for digital neuromorphic hardware, NOSONets trained using BPLC likely reduce inference delay significantly. To identify the feasibility of BPLC + NOSO, we trained CNN-based NOSONets on Fashion-MNIST and CIFAR-10. The classification accuracy on CIFAR-10 exceeds the state-of-the-art result from an SNN of the same depth and width by approximately 2%. Additionally, the number of spikes for inference is significantly reduced (by approximately one order of magnitude), highlighting a significant reduction in inference delay.
Paper: https://link.springer.com/article/10.1007/s40747-023-00983-y
Code: https://github.com/dooseokjeong/BPLC-NOSO
We have a new a new snnTorch tutorial/notebook on training spiking neurons with varied types of recurrent spiking neurons.
This is the second tutorial in a 3-part series on regression.
Link to the tutorial here.
Abstract: The vast majority of studies that process and analyze neural signals are conducted on cloud computing resources, which is often necessary for the demanding requirements of Deep Neural Network (DNN) workloads. However, applications such as epileptic seizure detection stand to benefit from edge devices that can securely analyze sensitive medical data in real-time and personalised manner. In this work, we propose a novel neuromorphic computing approach to seizure detection using a surrogate gradient-based deep Spiking Neural Network (SNN), which consists of a novel Spiking ConvLSTM unit (SPCLU). We have trained, validated, and rigorously tested the proposed SNN model across three publicly accessible datasets, including Boston Children’s Hospital–MIT (CHB-MIT) dataset from the U.S., and the Freiburg (FB) and EPILEPSIAE intracranial EEG (iEEG) datasets from Germany. The average leave-one-out cross-validation AUC score for FB, CHB-MIT, and EPILEPSIAE datasets can reach 92.7%, 89.0%, and 81.1%, respectively, while the computational overhead and energy consumption are significantly reduced when compared to alternative state-of-the-art models, showing the potential for building an accurate hardware-friendly, low-power neuromorphic system. This is the first feasibility study using a deep Spiking Neural Network for seizure detection on several reliable public datasets.
Read more here.
Farhad Modaresi has led the design and tape-out of a fully open-sourced spiking neural network accelerator in the Skywater 130 process. The design is based on memory macros generated using OpenRAM.
Many of the advances in deep learning this past decade can be attributed to the open-source movement where researchers have been able to reproduce and iterate upon open code bases. With the advent of open PDKs (SkyWater), EDA toolchains, and memory compilers (OpenRAM by co-author Matthew Guthaus), we hope to port rapid acceleration in hardware development to the neuromorphic community.
Check out the preprint here: https://arxiv.org/abs/2302.01015
GitHub repo with RTL you are welcome to steal: https://github.com/sfmth/OpenSpike
We have a new a new snnTorch tutorial/notebook on nonlinear regression with SNNs.
This first part will show you how to train a spiking neuron’s membrane potential to follow a given trajectory over time. Future tutorials will introduce spiking LSTM models, recurrent leaky integrator neurons, and do some more fancy regression tasks.
Link to the tutorial here.
Nonlinear Regression with SNNs
Ph.D. Candidate Peng Zhou who recently had a successful dissertation defense has published her latest work that applies gradient-based learning to the naturally arising spiking dynamics that are present in spiking neurons. Read the full paper here.
Vincent Sun and I have been working with Graphcore to speed-up spiking neural net training workloads on deep learning accelerators.
We’ve integrated snnTorch with Intelligence Processing Units (Graphcore) that leverage sparsity and MIMD execution by a factor of 5-10x over GPUs.
We have released a preprint that describes what we did, a rigorous analysis of IPUs vs GPUs for gradient-based SNN training across a variety of benchmarks in SNNs across neuron models, architectures, and propose a few additional tricks to speed up training (e.g., population codes). We also stress-tested the IPUs and found a few corner cases where GPU performance is on-par (recurrent SNNs with 1-1 connectivity).
Preprint: https://lnkd.in/gaTGjjh3
Code: https://lnkd.in/gTQ2SE7q
Read more here.
In collaboration with Prof. Ramtin Zand, Mohammed Elbtity, Peyton S. Chandrana, and Brendan Reidy at the iCAS lab. Read more here.
Read more about the Telluride Neuromorphic Cognition workshop here.
We now have snnTorch accelerated on Graphcore’s Intelligent Processing Units (IPUs), co-developed by Vincent Sun, Jason Eshraghian, along with assistance from Graphcore.
See this link for a tutorial to train SNNs on IPUs.
