Event-based backpropagation can compute exact gradients for spiking neural networks
Spiking neural networks combine analog computation with event-based communication using discrete spikes. While the impressive advances of deep learning are enabled by training non-spiking artificial neural networks using the backpropagation algorithm, applying this algorithm to spiking networks was...
Gespeichert in:
| Hauptverfasser: | , |
|---|---|
| Dokumenttyp: | Article (Journal) |
| Sprache: | Englisch |
| Veröffentlicht: |
18 June 2021
|
| In: |
Scientific reports
Year: 2021, Jahrgang: 11, Pages: 1-17 |
| ISSN: | 2045-2322 |
| DOI: | 10.1038/s41598-021-91786-z |
| Online-Zugang: | Verlag, lizenzpflichtig, Volltext: https://doi.org/10.1038/s41598-021-91786-z Verlag, lizenzpflichtig, Volltext: https://www.nature.com/articles/s41598-021-91786-z 
|
| Verfasserangaben: | Timo C. Wunderlich & Christian Pehle |
MARC
| LEADER | 00000caa a2200000 c 4500 | ||
|---|---|---|---|
| 001 | 1777951720 | ||
| 003 | DE-627 | ||
| 005 | 20220820080452.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 211118s2021 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1038/s41598-021-91786-z |2 doi | |
| 035 | |a (DE-627)1777951720 | ||
| 035 | |a (DE-599)KXP1777951720 | ||
| 035 | |a (OCoLC)1341430838 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 29 |2 sdnb | ||
| 100 | 1 | |a Wunderlich, Timo |e VerfasserIn |0 (DE-588)1187260010 |0 (DE-627)1666373451 |4 aut | |
| 245 | 1 | 0 | |a Event-based backpropagation can compute exact gradients for spiking neural networks |c Timo C. Wunderlich & Christian Pehle |
| 264 | 1 | |c 18 June 2021 | |
| 300 | |a 17 | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 520 | |a Spiking neural networks combine analog computation with event-based communication using discrete spikes. While the impressive advances of deep learning are enabled by training non-spiking artificial neural networks using the backpropagation algorithm, applying this algorithm to spiking networks was previously hindered by the existence of discrete spike events and discontinuities. For the first time, this work derives the backpropagation algorithm for a continuous-time spiking neural network and a general loss function by applying the adjoint method together with the proper partial derivative jumps, allowing for backpropagation through discrete spike events without approximations. This algorithm, EventProp, backpropagates errors at spike times in order to compute the exact gradient in an event-based, temporally and spatially sparse fashion. We use gradients computed via EventProp to train networks on the Yin-Yang and MNIST datasets using either a spike time or voltage based loss function and report competitive performance. Our work supports the rigorous study of gradient-based learning algorithms in spiking neural networks and provides insights toward their implementation in novel brain-inspired hardware. | ||
| 650 | 4 | |a Learning algorithms | |
| 650 | 4 | |a Machine learning | |
| 650 | 4 | |a Mathematics and computing | |
| 700 | 1 | |a Pehle, Christian |d 1988- |e VerfasserIn |0 (DE-588)1093238550 |0 (DE-627)853202338 |0 (DE-576)46265527X |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Scientific reports |d [London] : Springer Nature, 2011 |g 11(2021), Artikel-ID 12829, Seite 1-17 |h Online-Ressource |w (DE-627)663366712 |w (DE-600)2615211-3 |w (DE-576)346641179 |x 2045-2322 |7 nnas |a Event-based backpropagation can compute exact gradients for spiking neural networks |
| 773 | 1 | 8 | |g volume:11 |g year:2021 |g elocationid:12829 |g pages:1-17 |g extent:17 |a Event-based backpropagation can compute exact gradients for spiking neural networks |
| 856 | 4 | 0 | |u https://doi.org/10.1038/s41598-021-91786-z |x Verlag |x Resolving-System |z lizenzpflichtig |3 Volltext |
| 856 | 4 | 0 | |u https://www.nature.com/articles/s41598-021-91786-z |x Verlag |z lizenzpflichtig |3 Volltext |
| 951 | |a AR | ||
| 992 | |a 20211118 | ||
| 993 | |a Article | ||
| 994 | |a 2021 | ||
| 998 | |g 1093238550 |a Pehle, Christian |m 1093238550:Pehle, Christian |d 700000 |d 728500 |e 700000PP1093238550 |e 728500PP1093238550 |k 0/700000/ |k 1/700000/728500/ |p 2 |y j | ||
| 998 | |g 1187260010 |a Wunderlich, Timo |m 1187260010:Wunderlich, Timo |p 1 |x j | ||
| 999 | |a KXP-PPN1777951720 |e 400333941X | ||
| BIB | |a Y | ||
| SER | |a journal | ||
| JSO | |a {"name":{"displayForm":["Timo C. Wunderlich & Christian Pehle"]},"person":[{"given":"Timo","family":"Wunderlich","role":"aut","roleDisplay":"VerfasserIn","display":"Wunderlich, Timo"},{"role":"aut","display":"Pehle, Christian","roleDisplay":"VerfasserIn","given":"Christian","family":"Pehle"}],"id":{"doi":["10.1038/s41598-021-91786-z"],"eki":["1777951720"]},"origin":[{"dateIssuedDisp":"18 June 2021","dateIssuedKey":"2021"}],"title":[{"title":"Event-based backpropagation can compute exact gradients for spiking neural networks","title_sort":"Event-based backpropagation can compute exact gradients for spiking neural networks"}],"language":["eng"],"relHost":[{"title":[{"title_sort":"Scientific reports","title":"Scientific reports"}],"recId":"663366712","language":["eng"],"type":{"bibl":"periodical","media":"Online-Ressource"},"disp":"Event-based backpropagation can compute exact gradients for spiking neural networksScientific reports","note":["Gesehen am 12.07.24"],"part":{"pages":"1-17","year":"2021","extent":"17","volume":"11","text":"11(2021), Artikel-ID 12829, Seite 1-17"},"pubHistory":["1, article number 1 (2011)-"],"id":{"zdb":["2615211-3"],"eki":["663366712"],"issn":["2045-2322"]},"origin":[{"publisherPlace":"[London] ; London","dateIssuedDisp":"2011-","dateIssuedKey":"2011","publisher":"Springer Nature ; Nature Publishing Group"}],"physDesc":[{"extent":"Online-Ressource"}]}],"recId":"1777951720","physDesc":[{"extent":"17 S."}],"type":{"media":"Online-Ressource","bibl":"article-journal"}} | ||
| SRT | |a WUNDERLICHEVENTBASED1820 | ||