Category Archives: Presentations

Prof. Jason Eshraghian Delivering Keynote at “Workshop on Synchronization and Timing Systems” – “The Brain Computes Using Time and so Should Neural Networks”

Posted on by

See the agenda here.

Abstract: How can “time” be harnessed to boost neural network performance? The brain is a marvel of computation and memory, processing vast amounts of sensory data with an efficiency that puts modern electronics to shame. Reducing the megawatts consumed by hyperscale datacenters to the mere 10 watts the brain requires demands a fundamental shift – leveraging time. We will explore how temporal dynamics enhance neural network efficiency and performance. We will explore the Matrix-Multiply-free language model, where information is distributed across sequences, requiring the model to “learn to forget” in order to utilize limited cache effectively. Ultimately, by embracing temporal strategies, we pave the way toward neuromorphic computing systems that are not only more efficient but also closer to the elegant and sustainable designs found in nature. This exploration marks a step forward in reducing energy demands while advancing the capabilities of artificial intelligence.

Prof. Jason Eshraghian Delivering Plenary Talk at IEEE MCSoC: “Large-Scale Neuromorphic Computing on Heterogeneous Systems”

Posted on by

In the realm of large-scale model training, the efficiency bottleneck often stems from the intensive data communication required between GPUs. Drawing inspiration from the brain’s remarkable efficiency, this talk explores neuromorphic computing’s potential to mitigate this bottleneck. As chip designers increasingly turn to advanced packaging technologies and chiplets, the models running on these heterogeneous platforms must evolve accordingly. Spiking neural networks, inspired by the brain’s method of encoding information over time and its utilization of fine-grained sparsity for information transfer, are perfectly poised to extract the benefits (and limitations) imposed in heterogeneous hardware systems. This talk will delve into strategies for integrating spiking neural networks into large-scale models and how neuromorphic computing, alongside the utilization of chiplets, can surpass the current capabilities of GPUs, paving the way for the next generation of AI systems.

Prof. Jason Eshraghian Delivers Invited Talk at Memrisys 2024: “A Pathway to Large-Scale Neuromorphic Memristive Systems”

Posted on by

Abstract:

Memristors and neuromorphic computing go together like spaghetti and meatballs. Their
promise of reaching brain-scale computational efficiency has significant implications for
accelerating cognitive workloads, so why haven’t we yet toppled NVIDIA from their throne?
While consultants might say it’s because of the lack of market inertia, and engineers might tell
you there are still technical hurdles to overcome. This talk will focus on the technical challenges
faced by circuit designers using memristors, specifically in the context of accelerating large-scale
deep learning workloads. These challenges are well-established, and treated as design
constraints in memristive circuits that presently exist. But overcoming those barriers remains an
open question. This talk provides a guide on how we might overcome their challenges using
systems-level approaches, and how spike-based computing could potentially be the right
problem for memristive computing, ultimately pushing past what have historically been
perceived as limitations.

Prof. Jason Eshraghian Delivering an Educational Class at 2024 Embedded Systems Week

Posted on by

What do Transformers have to learn from Biological Spiking Neural Networks?

The brain is the perfect place to look for inspiration to develop more efficient neural networks. One of the main differences with modern deep learning is that the brain encodes and processes information as spikes rather than continuous, high-precision activations. This presentation will dive into how the open-source ecosystem has been used to develop brain-inspired neuromorphic accelerators, from our development of a Python training library for spiking neural networks (snnTorch, >100,000 downloads). We will explore how this is linked to our MatMul-free Language Model, providing insight into the next generation of large-scale, billion parameter models.

“Knowledge Distillation Through Time for Future Event Prediction” Presented at ICLR by Undergraduate Researcher Skye Gunasekaran

Posted on by

Abstract:  Is it possible to learn from the future? Here, we introduce knowledge distillation through time (KDTT). In traditional knowledge distillation (KD), a reliable teacher model is used to train an error-prone student model. The difference between the teacher and student is typically model capacity; the teacher is larger in architecture. In KDTT, the teacher and student models differ in their assigned tasks. The teacher model is tasked with detecting events in sequential data, a simple task compared to the student model, which is challenged with forecasting said events in the future. Through KDTT, the student can use the ’future’ logits from a teacher model to extract temporal uncertainty. We show the efficacy of KDTT on seizure prediction, where the student forecaster achieves a 20.0% average increase in the area under the curve of the receiver operating characteristic (AUC-ROC)

Prof. Jason Eshraghian and Dr. Fabrizio Ottati Present Tutorial at ISFPGA (Monterey, CA)

Posted on by

Fabrizio Ottati and I will be running a tutorial tomorrow (Sunday, 3 March) at the International Symposium on Field-Programmable Gate Arrays (ISFPGA) in Monterey, CA titled: “Who needs neuromorphic hardware? Deploying SNNs to FPGAs via HLS”.

snn-to-fpga

We’ll go through software and hardware: training SNNs using quantization-aware techniques across weights and stateful quantization, and then show how to go from an snnTorch model straight into AMD/Xilinx FPGAs for low-power + flexible deployment.

GitHub repo: https://github.com/open-neuromorphic/fpga-snntorch

Tutorial summary: https://www.isfpga.org/workshops-tutorials/#t2

OpenNeuromorphic Talk: “Neuromorphic Intermediate Representation”

Posted on by

In this workshop, we will show you how to move models from your favourite framework directly to neuromorphic hardware with 1-2 lines of code. We will present the technology behind, the Neuromorphic Intermediate Representation , and demonstrate how we can use it to run a live spiking convnet on the Speck chip.
Presented by Jens Pedersen, Bernhard Vogginger, Felix Bauer, and Jason Eshraghian. See the recording here.

Invited Talk: Kraw Lecture Series “Bridging the Gap Between Artificial Intelligence and Natural Intelligence” by Prof. Jason Eshraghian

Posted on by

See the recording here.

The Kraw Lecture Series in Silicon Valley is made possible by a generous gift from UC Santa Cruz alumnus George Kraw (Cowell ‘71, history and Russian literature) and Raphael Shannon Kraw. The lecture series features acclaimed UC Santa Cruz scientists and technologists who are grappling with some of the biggest questions of our time.

Abstract: The brain is the perfect place to look for inspiration to develop more efficient neural networks. Indeed, the inner workings of our synapses and neurons offer a glimpse at what the future of deep learning might look like. Our brains are constantly adapting, our neurons processing all that we know, mistakes we’ve made, failed predictions—all working to anticipate what will happen next with incredible speed. Our brains are also amazingly efficient. Training large-scale neural networks can cost more than $10 million in energy expense, yet the human brain does remarkably well on a power budget of 20 watts.

We can apply the computational principles that underpin the brain, and use them to engineer more efficient systems that adapt to ever changing environments. There is an interplay between neural inspired algorithms, how they can be deployed on low-power microelectronics, and how the brain provides a blueprint for this process.

Prof. Jason Eshraghian, Prof. Charlotte Frenkel and Prof. Rajit Manohar Present Tutorial at ESSCIRC/ESSDIRC (Lisbon, Portugal)

Posted on by

The tutorial titled “Open-Source Neuromorphic Circuit Design” ran in-person at the IEEE European Solid-State Circuits/Devices Conference, alongside co-presenters Prof. Charlotte Frenkel (TU Delft) and Prof. Rajit Manohar (Yale University). A live demo session with the notebooks that I’ll go through have been uploaded to GitHub at this link.

Tutorial Overview: As a bio-inspired alternative to conventional machine-learning accelerators, neuromorphic circuits outline promising energy savings for extreme-edge scenarios. While still being considered as an emerging approach, neuromorphic chip design is now being included in worldwide research roadmaps: the community is growing fast and is currently catalyzed by the development of open-source design tools and platforms. In this tutorial, we will survey the diversity of the open-source neuromorphic chip design landscape, from digital and mixed-signal small-scale proofs-of-concept to large-scale platforms. We will also provide a hands-on overview of the associated design challenges and guidelines, from which we will extract upcoming trends and promising use cases.

Prof. Jason Eshraghian Presenting Invited Talk at the Memristec Summer School, Dresden, Germany

Posted on by

Prof. Jason Eshraghian will be presented “A Hands-On Approach to Open-Source Memristive Neuromorphic Systems” at the DFG priority programme Memristec Summer School in Dresden, Germany.

Thsi interdisciplinary Summer School on memristive systems will bring students together and enable discussion with experts from the fields of fabrication, characterization and the theory of memristors.

 

Prof. Jason Eshraghian and Fabrizio Ottati to Present Tutorial at ICONS 2023 (Santa Fe, NM, USA)

Posted on by

The tutorial titled “From Training to Deployment: Build a Spiking FPGA on a $300 Budget” will run in-person at the International Conference on Neuromorphic Systems in Santa Fe, NM, USA.

Tutorial Abstract: 

So you want to use natural intelligence to improve artificial intelligence? The human brain is a great place to look to improve modern neural networks. The computational cost of deep learning exceeds millions of dollars to train large-scale models, and yet, our brains somehow achieve remarkable feats within a power budget of approximately 10-20 watts. While we may be far from having a complete understanding of the brain, we are at a point where a set of design principles has enabled us to build ultra-efficient deep learning tools. Most of these are linked back to event-driven spiking neural networks (SNNs).

In a cruel twist of irony, most of our SNNs are trained and battle-tested on GPUs. GPUs are far from optimized for spike-based workloads. The neuromorphic hardware that is out there for research and/or commercial use (a lot of love going to you, Intel Labs and SynSense), is considerably more expensive than a consumer-grade GPU, or require a few more steps than a single-click purchase off Amazon. The drawback of rich feature-sets is that at some point in the abstraction, such tools become inflexible. How can we move towards using low-cost hardware that sits on our desk, or fits in a PCIe slot in our desktops, and accelerates SNNs?

This tutorial will take a hands-on approach to learning how to train SNNs for hardware deployment, and running these models on a low-cost FPGA for inference. With the advent of open-sourced neuromorphic training libraries and electronic design automation tools, we will conduct hands-on coding sessions to train SNNs, and attendees will subsequently learn how to deploy the design to off the shelf hardware (namely, FPGAs), using the AMD Xilinx Kria KV260 starter kit as the accelerating platform. To port the SNN model to the hardware platform, a software tool being developed by Mr. Fabrizio Ottati, hls4nm, will be used, showing how to reap the rewards of training SNNs using hardware that you can own, break apart, and can put back together.

Telluride Workshop: Open Source Neuromorphic Hardware, Software and Wetware

Posted on by

Prof. Jason Eshraghian & Dr. Peng Zhou were topic area leaders at the Telluride Neuromorphic Engineering & Cognition Workshop. Tasks addressed included:

A project highlight includes the development of the Neuromorphic Intermediate Representation (NIR), an intermediate representation to translate various neuromorphic and physics-driven models that are based on continuous time ODEs into different formats. This makes it much easier to deploy models trained in one library to map to a large variety of backends.

Ruijie Zhu and Prof. Jason Eshraghian Present Invited Talk “Scaling up SNNs with SpikeGPT” at the Intel Neuromorphic Research Centre

Posted on by

spikegpt-architecture

Abstract: If we had a dollar for every time we heard “It will never scale!”, then neuromorphic engineers would be billionaires. This presentation will be centered on SpikeGPT, the first large-scale language model (LLM) using spiking neural nets (SNNs), and possibly the largest SNN that has been trained using error backpropagation.

The need for lightweight language models is more pressing than ever, especially now that we are becoming increasingly reliant on them from word processors and search engines, to code troubleshooting and academic grant writing. Our dependence on a single LLM means that every user is potentially pooling sensitive data into a singular database, which leads to significant security risks if breached.

SpikeGPT was built to move towards addressing the privacy and energy consumption challenges we presently run into using Transformer blocks. Our approach decomposes self-attention down into a recurrent form that is compatible with spiking neurons, along with dynamical weight matrices where the dynamics are learnable, rather than the parameters as with conventional deep learning.

We will provide an overview of what SpikeGPT does, how it works, and what it took to train it successfully. We will also provide a demo on how users can download pre-trained models available on HuggingFace so that listeners are able to experiment with them.

Link to the talk can be found here.