Essentially no barriers in neural network energy landscape
Training neural networks involves finding minima of a high-dimensional non-convex loss function. Knowledge of the structure of this energy landscape is sparse. Relaxing from linear interpolations, we construct continuous paths between minima of recent neural network architectures on CIFAR10 and CIFA...
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Dokumenttyp: | Article (Journal) Chapter/Article |
| Sprache: | Englisch |
| Veröffentlicht: |
2 Mar 2018
|
| In: |
Arxiv
|
| Online-Zugang: | kostenfrei
|
| Verfasserangaben: | Felix Draxler, Kambis Veschgini, Manfred Salmhofer, Fred A. Hamprecht |
MARC
| LEADER | 00000caa a2200000 4500 | ||
|---|---|---|---|
| 001 | 1570671567 | ||
| 003 | DE-627 | ||
| 005 | 20241030122218.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 180309s2018 xx |||||o 00| ||eng c | ||
| 035 | |a (DE-627)1570671567 | ||
| 035 | |a (DE-576)500671567 | ||
| 035 | |a (DE-599)BSZ500671567 | ||
| 035 | |a (OCoLC)1340994027 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 29 |2 sdnb | ||
| 100 | 1 | |a Draxler, Felix |d 1993- |e VerfasserIn |0 (DE-588)1223470296 |0 (DE-627)1742833683 |4 aut | |
| 245 | 1 | 0 | |a Essentially no barriers in neural network energy landscape |c Felix Draxler, Kambis Veschgini, Manfred Salmhofer, Fred A. Hamprecht |
| 264 | 1 | |c 2 Mar 2018 | |
| 300 | |a 12 | ||
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Identifizierung der Ressource nach: Last revised 22 Feb 2019 | ||
| 500 | |a Gesehen am 15.12.2020 | ||
| 520 | |a Training neural networks involves finding minima of a high-dimensional non-convex loss function. Knowledge of the structure of this energy landscape is sparse. Relaxing from linear interpolations, we construct continuous paths between minima of recent neural network architectures on CIFAR10 and CIFAR100. Surprisingly, the paths are essentially flat in both the training and test landscapes. This implies that neural networks have enough capacity for structural changes, or that these changes are small between minima. Also, each minimum has at least one vanishing Hessian eigenvalue in addition to those resulting from trivial invariance. | ||
| 650 | 4 | |a Computer Science - Artificial Intelligence | |
| 650 | 4 | |a Computer Science - Learning | |
| 650 | 4 | |a Statistics - Machine Learning | |
| 700 | 1 | |a Veschgini, Kambis |e VerfasserIn |0 (DE-588)1052119220 |0 (DE-627)787846155 |0 (DE-576)407868798 |4 aut | |
| 700 | 1 | |a Salmhofer, Manfred |d 1964- |e VerfasserIn |0 (DE-588)12037868X |0 (DE-627)080636934 |0 (DE-576)179574744 |4 aut | |
| 700 | 1 | |a Hamprecht, Fred |e VerfasserIn |0 (DE-588)1020505605 |0 (DE-627)691240280 |0 (DE-576)360605516 |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Arxiv |d Ithaca, NY : Cornell University, 1991 |g (2018) Artikel-Nummer 1803.00885, 12 Seiten |h Online-Ressource |w (DE-627)509006531 |w (DE-600)2225896-6 |w (DE-576)28130436X |7 nnns |a Essentially no barriers in neural network energy landscape |
| 773 | 1 | 8 | |g year:2018 |g extent:12 |a Essentially no barriers in neural network energy landscape |
| 856 | 4 | 0 | |u http://arxiv.org/abs/1803.00885 |x Verlag |z kostenfrei |3 Volltext |
| 951 | |a AR | ||
| 992 | |a 20180309 | ||
| 993 | |a Article | ||
| 994 | |a 2018 | ||
| 998 | |g 1020505605 |a Hamprecht, Fred |m 1020505605:Hamprecht, Fred |d 700000 |d 708070 |e 700000PH1020505605 |e 708070PH1020505605 |k 0/700000/ |k 1/700000/708070/ |p 4 |y j | ||
| 998 | |g 12037868X |a Salmhofer, Manfred |m 12037868X:Salmhofer, Manfred |d 130000 |d 130300 |e 130000PS12037868X |e 130300PS12037868X |k 0/130000/ |k 1/130000/130300/ |p 3 | ||
| 998 | |g 1052119220 |a Veschgini, Kambis |m 1052119220:Veschgini, Kambis |d 130000 |d 130300 |e 130000PV1052119220 |e 130300PV1052119220 |k 0/130000/ |k 1/130000/130300/ |p 2 | ||
| 998 | |g 1223470296 |a Draxler, Felix |m 1223470296:Draxler, Felix |d 130000 |e 130000PD1223470296 |k 0/130000/ |p 1 |x j | ||
| 999 | |a KXP-PPN1570671567 |e 3002150509 | ||
| BIB | |a Y | ||
| JSO | |a {"relHost":[{"note":["Gesehen am 28.05.2024"],"part":{"year":"2018","text":"(2018) Artikel-Nummer 1803.00885, 12 Seiten","extent":"12"},"disp":"Essentially no barriers in neural network energy landscapeArxiv","id":{"zdb":["2225896-6"],"eki":["509006531"]},"pubHistory":["1991 -"],"language":["eng"],"recId":"509006531","type":{"media":"Online-Ressource","bibl":"edited-book"},"titleAlt":[{"title":"Arxiv.org"},{"title":"Arxiv.org e-print archive"},{"title":"Arxiv e-print archive"},{"title":"De.arxiv.org"}],"origin":[{"publisherPlace":"Ithaca, NY ; [Erscheinungsort nicht ermittelbar]","publisher":"Cornell University ; Arxiv.org","dateIssuedKey":"1991","dateIssuedDisp":"1991-"}],"physDesc":[{"extent":"Online-Ressource"}],"title":[{"title_sort":"Arxiv","title":"Arxiv"}]}],"note":["Identifizierung der Ressource nach: Last revised 22 Feb 2019","Gesehen am 15.12.2020"],"origin":[{"dateIssuedDisp":"2 Mar 2018","dateIssuedKey":"2018"}],"id":{"eki":["1570671567"]},"language":["eng"],"recId":"1570671567","type":{"bibl":"chapter","media":"Online-Ressource"},"person":[{"given":"Felix","display":"Draxler, Felix","family":"Draxler","role":"aut","roleDisplay":"VerfasserIn"},{"role":"aut","roleDisplay":"VerfasserIn","given":"Kambis","display":"Veschgini, Kambis","family":"Veschgini"},{"display":"Salmhofer, Manfred","family":"Salmhofer","given":"Manfred","role":"aut","roleDisplay":"VerfasserIn"},{"given":"Fred","family":"Hamprecht","display":"Hamprecht, Fred","role":"aut","roleDisplay":"VerfasserIn"}],"title":[{"title":"Essentially no barriers in neural network energy landscape","title_sort":"Essentially no barriers in neural network energy landscape"}],"physDesc":[{"extent":"S.12"}],"name":{"displayForm":["Felix Draxler, Kambis Veschgini, Manfred Salmhofer, Fred A. Hamprecht"]}} | ||
| SRT | |a DRAXLERFELESSENTIALL2201 | ||