Cover Image

Sampling-free variational inference of Bayesian neural networks by variance backpropagation

We propose a new Bayesian Neural Net formulation that affords variational inference for which the evidence lower bound is analytically tractable subject to a tight approximation. We achieve this tractability by (i) decomposing ReLU nonlinearities into the product of an identity and a Heaviside step...

Full description

Saved in:
Bibliographic Details
Main Authors: Haußmann, Manuel (Author) , Hamprecht, Fred (Author) , Kandemir, Melih (Author)
Format: Article (Journal) Chapter/Article
Language:English
Published: 2018
In: Arxiv
Year: 2018, Pages: 1-15
DOI:10.48550/arXiv.1805.07654
Online Access:Verlag, lizenzpflichtig, Volltext: https://doi.org/10.48550/arXiv.1805.07654
Verlag, lizenzpflichtig, Volltext: http://arxiv.org/abs/1805.07654
Get full text
Author Notes:Manuel Haußmann, Fred A. Hamprecht, Melih Kandemir

MARC

LEADER 00000caa a2200000 c 4500
001 181009206X
003 DE-627
005 20220820222716.0
007 cr uuu---uuuuu
008 220713s2018 xx |||||o 00| ||eng c
024 7 |a 10.48550/arXiv.1805.07654  |2 doi 
035 |a (DE-627)181009206X 
035 |a (DE-599)KXP181009206X 
035 |a (OCoLC)1341463814 
040 |a DE-627  |b ger  |c DE-627  |e rda 
041 |a eng 
084 |a 29  |2 sdnb 
100 1 |a Haußmann, Manuel  |d 1990-  |e VerfasserIn  |0 (DE-588)1205492046  |0 (DE-627)1691155802  |4 aut 
245 1 0 |a Sampling-free variational inference of Bayesian neural networks by variance backpropagation  |c Manuel Haußmann, Fred A. Hamprecht, Melih Kandemir 
264 1 |c 2018 
300 |a 15 
336 |a Text  |b txt  |2 rdacontent 
337 |a Computermedien  |b c  |2 rdamedia 
338 |a Online-Ressource  |b cr  |2 rdacarrier 
500 |a Last revised 12 Jun 2019 
500 |a Gesehen am 13.07.2022 
520 |a We propose a new Bayesian Neural Net formulation that affords variational inference for which the evidence lower bound is analytically tractable subject to a tight approximation. We achieve this tractability by (i) decomposing ReLU nonlinearities into the product of an identity and a Heaviside step function, (ii) introducing a separate path that decomposes the neural net expectation from its variance. We demonstrate formally that introducing separate latent binary variables to the activations allows representing the neural network likelihood as a chain of linear operations. Performing variational inference on this construction enables a sampling-free computation of the evidence lower bound which is a more effective approximation than the widely applied Monte Carlo sampling and CLT related techniques. We evaluate the model on a range of regression and classification tasks against BNN inference alternatives, showing competitive or improved performance over the current state-of-the-art. 
650 4 |a Computer Science - Machine Learning 
650 4 |a Statistics - Machine Learning 
700 1 |a Hamprecht, Fred  |e VerfasserIn  |0 (DE-588)1020505605  |0 (DE-627)691240280  |0 (DE-576)360605516  |4 aut 
700 1 |a Kandemir, Melih  |d 1983-  |e VerfasserIn  |0 (DE-588)1067700463  |0 (DE-627)819103551  |0 (DE-576)426867181  |4 aut 
773 0 8 |i Enthalten in  |t Arxiv  |d Ithaca, NY : Cornell University, 1991  |g (2018), Artikel-ID 1805.07654, Seite 1-15  |h Online-Ressource  |w (DE-627)509006531  |w (DE-600)2225896-6  |w (DE-576)28130436X  |7 nnas  |a Sampling-free variational inference of Bayesian neural networks by variance backpropagation 
773 1 8 |g year:2018  |g elocationid:1805.07654  |g pages:1-15  |g extent:15  |a Sampling-free variational inference of Bayesian neural networks by variance backpropagation 
856 4 0 |u https://doi.org/10.48550/arXiv.1805.07654  |x Verlag  |x Resolving-System  |z lizenzpflichtig  |3 Volltext 
856 4 0 |u http://arxiv.org/abs/1805.07654  |x Verlag  |z lizenzpflichtig  |3 Volltext 
951 |a AR 
992 |a 20220713 
993 |a Article 
994 |a 2018 
998 |g 1020505605  |a Hamprecht, Fred  |m 1020505605:Hamprecht, Fred  |d 700000  |d 708070  |e 700000PH1020505605  |e 708070PH1020505605  |k 0/700000/  |k 1/700000/708070/  |p 2 
998 |g 1205492046  |a Haußmann, Manuel  |m 1205492046:Haußmann, Manuel  |d 700000  |d 708070  |e 700000PH1205492046  |e 708070PH1205492046  |k 0/700000/  |k 1/700000/708070/  |p 1  |x j 
999 |a KXP-PPN181009206X  |e 4165160946 
BIB |a Y 
JSO |a {"relHost":[{"recId":"509006531","pubHistory":["1991 -"],"physDesc":[{"extent":"Online-Ressource"}],"id":{"eki":["509006531"],"zdb":["2225896-6"]},"disp":"Sampling-free variational inference of Bayesian neural networks by variance backpropagationArxiv","language":["eng"],"part":{"extent":"15","year":"2018","text":"(2018), Artikel-ID 1805.07654, Seite 1-15","pages":"1-15"},"note":["Gesehen am 28.05.2024"],"origin":[{"dateIssuedKey":"1991","publisher":"Cornell University ; Arxiv.org","publisherPlace":"Ithaca, NY ; [Erscheinungsort nicht ermittelbar]","dateIssuedDisp":"1991-"}],"type":{"bibl":"edited-book","media":"Online-Ressource"},"titleAlt":[{"title":"Arxiv.org"},{"title":"Arxiv.org e-print archive"},{"title":"Arxiv e-print archive"},{"title":"De.arxiv.org"}],"title":[{"title":"Arxiv","title_sort":"Arxiv"}]}],"id":{"eki":["181009206X"],"doi":["10.48550/arXiv.1805.07654"]},"name":{"displayForm":["Manuel Haußmann, Fred A. Hamprecht, Melih Kandemir"]},"physDesc":[{"extent":"15 S."}],"recId":"181009206X","title":[{"title_sort":"Sampling-free variational inference of Bayesian neural networks by variance backpropagation","title":"Sampling-free variational inference of Bayesian neural networks by variance backpropagation"}],"note":["Last revised 12 Jun 2019","Gesehen am 13.07.2022"],"origin":[{"dateIssuedDisp":"2018","dateIssuedKey":"2018"}],"type":{"media":"Online-Ressource","bibl":"chapter"},"language":["eng"],"person":[{"display":"Haußmann, Manuel","family":"Haußmann","role":"aut","given":"Manuel"},{"given":"Fred","role":"aut","family":"Hamprecht","display":"Hamprecht, Fred"},{"family":"Kandemir","display":"Kandemir, Melih","given":"Melih","role":"aut"}]} 
SRT |a HAUSSMANNMSAMPLINGFR2018 

Similar Items