Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Devansh Arpit

Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization


Dec 28, 2020
Stanislaw Jastrzebski, Devansh Arpit, Oliver Astrand, Giancarlo Kerg, Huan Wang, Caiming Xiong, Richard Socher, Kyunghyun Cho, Krzysztof Geras

* The last two authors contributed equally 

  Access Paper or Ask Questions

The Break-Even Point on Optimization Trajectories of Deep Neural Networks


Feb 21, 2020
Stanislaw Jastrzebski, Maciej Szymczak, Stanislav Fort, Devansh Arpit, Jacek Tabor, Kyunghyun Cho, Krzysztof Geras

* Accepted as a spotlight at ICLR 2020. The last two authors contributed equally 

  Access Paper or Ask Questions

Neural Bayes: A Generic Parameterization Method for Unsupervised Representation Learning


Feb 20, 2020
Devansh Arpit, Huan Wang, Caiming Xiong, Richard Socher, Yoshua Bengio


  Access Paper or Ask Questions

Entropy Penalty: Towards Generalization Beyond the IID Assumption


Oct 01, 2019
Devansh Arpit, Caiming Xiong, Richard Socher


  Access Paper or Ask Questions

How to Initialize your Network? Robust Initialization for WeightNorm & ResNets


Jun 05, 2019
Devansh Arpit, Victor Campos, Yoshua Bengio


  Access Paper or Ask Questions

The Benefits of Over-parameterization at Initialization in Deep ReLU Networks


Jan 11, 2019
Devansh Arpit, Yoshua Bengio


  Access Paper or Ask Questions

On the Spectral Bias of Neural Networks


Oct 17, 2018
Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred A. Hamprecht, Yoshua Bengio, Aaron Courville


  Access Paper or Ask Questions

Three Factors Influencing Minima in SGD


Sep 13, 2018
Stanisław Jastrzębski, Zachary Kenton, Devansh Arpit, Nicolas Ballas, Asja Fischer, Yoshua Bengio, Amos Storkey

* First two authors contributed equally. Short version accepted into ICLR workshop. Accepted to Artificial Neural Networks and Machine Learning, ICANN 2018 

  Access Paper or Ask Questions

A Walk with SGD


May 30, 2018
Chen Xing, Devansh Arpit, Christos Tsirigotis, Yoshua Bengio

* First two authors contributed equally 

  Access Paper or Ask Questions

Fraternal Dropout


Mar 28, 2018
Konrad Zolna, Devansh Arpit, Dendi Suhubdy, Yoshua Bengio

* Accepted to ICLR 2018. Extended appendix. Added official GitHub code for replication: https://github.com/kondiz/fraternal-dropout . Added references. Corrected typos 

  Access Paper or Ask Questions

Residual Connections Encourage Iterative Inference


Mar 08, 2018
Stanisław Jastrzębski, Devansh Arpit, Nicolas Ballas, Vikas Verma, Tong Che, Yoshua Bengio

* First two authors contributed equally. Published in ICLR 2018 

  Access Paper or Ask Questions

Variational Bi-LSTMs


Nov 15, 2017
Samira Shabanian, Devansh Arpit, Adam Trischler, Yoshua Bengio


  Access Paper or Ask Questions

On Optimality Conditions for Auto-Encoder Signal Recovery


Jul 13, 2017
Devansh Arpit, Yingbo Zhou, Hung Q. Ngo, Nils Napp, Venu Govindaraju


  Access Paper or Ask Questions

A Closer Look at Memorization in Deep Networks


Jul 01, 2017
Devansh Arpit, Stanisław Jastrzębski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S. Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, Simon Lacoste-Julien

* Appears in Proceedings of the 34th International Conference on Machine Learning (ICML 2017), Devansh Arpit, Stanis{\l}aw Jastrz\k{e}bski, Nicolas Ballas, and David Krueger contributed equally to this work 

  Access Paper or Ask Questions

Normalization Propagation: A Parametric Technique for Removing Internal Covariate Shift in Deep Networks


Jul 12, 2016
Devansh Arpit, Yingbo Zhou, Bhargava U. Kota, Venu Govindaraju

* 11 pages, ICML 2016, appendix added to the last version 

  Access Paper or Ask Questions

Why Regularized Auto-Encoders learn Sparse Representation?


Jun 17, 2016
Devansh Arpit, Yingbo Zhou, Hung Ngo, Venu Govindaraju

* 8 pages of content, 1 page of reference, 4 pages of supplementary. ICML 2016; bug fix in lemma 1 

  Access Paper or Ask Questions

Dimensionality Reduction with Subspace Structure Preservation


Apr 06, 2016
Devansh Arpit, Ifeoma Nwogu, Venu Govindaraju

* Published in NIPS 2014; v2: minor updates to the algorithm and added a few lines addressing application to large-scale/high-dimensional data 

  Access Paper or Ask Questions

Is Joint Training Better for Deep Auto-Encoders?


Jun 15, 2015
Yingbo Zhou, Devansh Arpit, Ifeoma Nwogu, Venu Govindaraju

* 11 pages, 4 figures 

  Access Paper or Ask Questions

An Analysis of Random Projections in Cancelable Biometrics


Nov 14, 2014
Devansh Arpit, Ifeoma Nwogu, Gaurav Srivastava, Venu Govindaraju


  Access Paper or Ask Questions