Vincent Sun and I have been working with Graphcore to speed-up spiking neural net training workloads on deep learning accelerators.

We’ve integrated snnTorch with Intelligence Processing Units (Graphcore) that leverage sparsity and MIMD execution by a factor of 5-10x over GPUs.

We have released a preprint that describes what we did, a rigorous analysis of IPUs vs GPUs for gradient-based SNN training across a variety of benchmarks in SNNs across neuron models, architectures, and propose a few additional tricks to speed up training (e.g., population codes). We also stress-tested the IPUs and found a few corner cases where GPU performance is on-par (recurrent SNNs with 1-1 connectivity).

Preprint: https://lnkd.in/gaTGjjh3

Code: https://lnkd.in/gTQ2SE7q