Identifying nonlinear dynamical systems with multiple time scales and long-range dependencies
A main theoretical interest in biology and physics is to identify the nonlinear dynamical system (DS) that generated observed time series. Recurrent Neural Networks (RNNs) are, in principle, powerful enough to approximate any underlying DS, but in their vanilla form suffer from the exploding vs. van...
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Dokumenttyp: | Kapitel/Artikel Konferenzschrift |
| Sprache: | Englisch |
| Veröffentlicht: |
12 Mar 2021
|
| Ausgabe: | Version v3 |
| In: |
Arxiv
Year: 2021, Pages: 1-29 |
| DOI: | 10.48550/arXiv.1910.03471 |
| Online-Zugang: | Verlag, kostenfrei, Volltext: https://doi.org/10.48550/arXiv.1910.03471 Verlag, kostenfrei, Volltext: http://arxiv.org/abs/1910.03471 
|
| Verfasserangaben: | Dominik Schmidt, Georgia Koppe, Zahra Monfared, Max Beutelspacher, Daniel Durstewitz |
MARC
| LEADER | 00000caa a2200000 c 4500 | ||
|---|---|---|---|
| 001 | 1818272873 | ||
| 003 | DE-627 | ||
| 005 | 20240110111247.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 221010s2021 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.48550/arXiv.1910.03471 |2 doi | |
| 035 | |a (DE-627)1818272873 | ||
| 035 | |a (DE-599)KXP1818272873 | ||
| 035 | |a (OCoLC)1361714125 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 28 |2 sdnb | ||
| 100 | 1 | |a Schmidt, Dominik |e VerfasserIn |0 (DE-588)1218815337 |0 (DE-627)1734538120 |4 aut | |
| 245 | 1 | 0 | |a Identifying nonlinear dynamical systems with multiple time scales and long-range dependencies |c Dominik Schmidt, Georgia Koppe, Zahra Monfared, Max Beutelspacher, Daniel Durstewitz |
| 250 | |a Version v3 | ||
| 264 | 1 | |c 12 Mar 2021 | |
| 300 | |b Illustrationen | ||
| 300 | |a 29 | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Online veröffentlicht am 8. Oktober 2019, Version 2 am 19. Juni 2020, Version 3 am 12. März 2021 | ||
| 500 | |a Gesehen am 10.01.2024 | ||
| 520 | |a A main theoretical interest in biology and physics is to identify the nonlinear dynamical system (DS) that generated observed time series. Recurrent Neural Networks (RNNs) are, in principle, powerful enough to approximate any underlying DS, but in their vanilla form suffer from the exploding vs. vanishing gradients problem. Previous attempts to alleviate this problem resulted either in more complicated, mathematically less tractable RNN architectures, or strongly limited the dynamical expressiveness of the RNN. Here we address this issue by suggesting a simple regularization scheme for vanilla RNNs with ReLU activation which enables them to solve long-range dependency problems and express slow time scales, while retaining a simple mathematical structure which makes their DS properties partly analytically accessible. We prove two theorems that establish a tight connection between the regularized RNN dynamics and its gradients, illustrate on DS benchmarks that our regularization approach strongly eases the reconstruction of DS which harbor widely differing time scales, and show that our method is also en par with other long-range architectures like LSTMs on several tasks. | ||
| 650 | 4 | |a Computer Science - Machine Learning | |
| 650 | 4 | |a Quantitative Biology - Quantitative Methods | |
| 650 | 4 | |a Statistics - Machine Learning | |
| 700 | 1 | |a Koppe, Georgia |d 1984- |e VerfasserIn |0 (DE-588)1095801198 |0 (DE-627)856418498 |0 (DE-576)467814724 |4 aut | |
| 700 | 1 | |a Monfared, Zahra |e VerfasserIn |0 (DE-588)1268472905 |0 (DE-627)1817003658 |4 aut | |
| 700 | 1 | |a Beutelspacher, Max |e VerfasserIn |4 aut | |
| 700 | 1 | |a Durstewitz, Daniel |d 1967- |e VerfasserIn |0 (DE-588)12042021X |0 (DE-627)080664008 |0 (DE-576)174757050 |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Arxiv |d Ithaca, NY : Cornell University, 1991 |g (2021), Artikel-ID 1910.03471, Seite 1-29 |h Online-Ressource |w (DE-627)509006531 |w (DE-600)2225896-6 |w (DE-576)28130436X |7 nnas |a Identifying nonlinear dynamical systems with multiple time scales and long-range dependencies |
| 773 | 1 | 8 | |g year:2021 |g elocationid:1910.03471 |g pages:1-29 |g extent:29 |a Identifying nonlinear dynamical systems with multiple time scales and long-range dependencies |
| 856 | 4 | 0 | |u https://doi.org/10.48550/arXiv.1910.03471 |x Verlag |x Resolving-System |z kostenfrei |3 Volltext |
| 856 | 4 | 0 | |u http://arxiv.org/abs/1910.03471 |x Verlag |z kostenfrei |3 Volltext |
| 951 | |a AR | ||
| 992 | |a 20221010 | ||
| 993 | |a ConferencePaper | ||
| 994 | |a 2021 | ||
| 998 | |g 12042021X |a Durstewitz, Daniel |m 12042021X:Durstewitz, Daniel |d 130000 |d 700000 |d 728500 |e 130000PD12042021X |e 700000PD12042021X |e 728500PD12042021X |k 0/130000/ |k 0/700000/ |k 1/700000/728500/ |p 5 |y j | ||
| 998 | |g 1268472905 |a Monfared, Zahra |m 1268472905:Monfared, Zahra |d 700000 |d 728500 |e 700000PM1268472905 |e 728500PM1268472905 |k 0/700000/ |k 1/700000/728500/ |p 3 | ||
| 998 | |g 1095801198 |a Koppe, Georgia |m 1095801198:Koppe, Georgia |d 60000 |e 60000PK1095801198 |k 0/60000/ |p 2 | ||
| 999 | |a KXP-PPN1818272873 |e 4195462053 | ||
| BIB | |a Y | ||
| JSO | |a {"recId":"1818272873","origin":[{"dateIssuedDisp":"12 Mar 2021","edition":"Version v3","dateIssuedKey":"2021"}],"type":{"bibl":"chapter","media":"Online-Ressource"},"physDesc":[{"noteIll":"Illustrationen","extent":"29 S."}],"id":{"eki":["1818272873"],"doi":["10.48550/arXiv.1910.03471"]},"relHost":[{"pubHistory":["1991 -"],"title":[{"title_sort":"Arxiv","title":"Arxiv"}],"language":["eng"],"part":{"pages":"1-29","extent":"29","text":"(2021), Artikel-ID 1910.03471, Seite 1-29","year":"2021"},"disp":"Identifying nonlinear dynamical systems with multiple time scales and long-range dependenciesArxiv","id":{"eki":["509006531"],"zdb":["2225896-6"]},"physDesc":[{"extent":"Online-Ressource"}],"note":["Gesehen am 28.05.2024"],"titleAlt":[{"title":"Arxiv.org"},{"title":"Arxiv.org e-print archive"},{"title":"Arxiv e-print archive"},{"title":"De.arxiv.org"}],"type":{"bibl":"edited-book","media":"Online-Ressource"},"origin":[{"publisherPlace":"Ithaca, NY ; [Erscheinungsort nicht ermittelbar]","dateIssuedDisp":"1991-","publisher":"Cornell University ; Arxiv.org","dateIssuedKey":"1991"}],"recId":"509006531"}],"note":["Online veröffentlicht am 8. Oktober 2019, Version 2 am 19. Juni 2020, Version 3 am 12. März 2021","Gesehen am 10.01.2024"],"name":{"displayForm":["Dominik Schmidt, Georgia Koppe, Zahra Monfared, Max Beutelspacher, Daniel Durstewitz"]},"person":[{"role":"aut","given":"Dominik","display":"Schmidt, Dominik","family":"Schmidt"},{"display":"Koppe, Georgia","role":"aut","given":"Georgia","family":"Koppe"},{"given":"Zahra","role":"aut","display":"Monfared, Zahra","family":"Monfared"},{"family":"Beutelspacher","role":"aut","given":"Max","display":"Beutelspacher, Max"},{"role":"aut","given":"Daniel","display":"Durstewitz, Daniel","family":"Durstewitz"}],"title":[{"title_sort":"Identifying nonlinear dynamical systems with multiple time scales and long-range dependencies","title":"Identifying nonlinear dynamical systems with multiple time scales and long-range dependencies"}],"language":["eng"]} | ||
| SRT | |a SCHMIDTDOMIDENTIFYIN1220 | ||