Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox
Picture for Devansh Arpit

Devansh Arpit

Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization


Dec 28, 2020
Stanislaw Jastrzebski, Devansh Arpit, Oliver Astrand, Giancarlo Kerg, Huan Wang, Caiming Xiong, Richard Socher, Kyunghyun Cho, Krzysztof Geras

Figure 1 for Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization
Figure 2 for Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization
Figure 3 for Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization
Figure 4 for Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization
* The last two authors contributed equally 
  Access Paper or Ask Questions

The Break-Even Point on Optimization Trajectories of Deep Neural Networks


Feb 21, 2020
Stanislaw Jastrzebski, Maciej Szymczak, Stanislav Fort, Devansh Arpit, Jacek Tabor, Kyunghyun Cho, Krzysztof Geras

Figure 1 for The Break-Even Point on Optimization Trajectories of Deep Neural Networks
Figure 2 for The Break-Even Point on Optimization Trajectories of Deep Neural Networks
Figure 3 for The Break-Even Point on Optimization Trajectories of Deep Neural Networks
Figure 4 for The Break-Even Point on Optimization Trajectories of Deep Neural Networks
* Accepted as a spotlight at ICLR 2020. The last two authors contributed equally 
  Access Paper or Ask Questions

Neural Bayes: A Generic Parameterization Method for Unsupervised Representation Learning


Feb 20, 2020
Devansh Arpit, Huan Wang, Caiming Xiong, Richard Socher, Yoshua Bengio

Figure 1 for Neural Bayes: A Generic Parameterization Method for Unsupervised Representation Learning
Figure 2 for Neural Bayes: A Generic Parameterization Method for Unsupervised Representation Learning
Figure 3 for Neural Bayes: A Generic Parameterization Method for Unsupervised Representation Learning
Figure 4 for Neural Bayes: A Generic Parameterization Method for Unsupervised Representation Learning

  Access Paper or Ask Questions

Entropy Penalty: Towards Generalization Beyond the IID Assumption


Oct 01, 2019
Devansh Arpit, Caiming Xiong, Richard Socher

Figure 1 for Entropy Penalty: Towards Generalization Beyond the IID Assumption
Figure 2 for Entropy Penalty: Towards Generalization Beyond the IID Assumption
Figure 3 for Entropy Penalty: Towards Generalization Beyond the IID Assumption
Figure 4 for Entropy Penalty: Towards Generalization Beyond the IID Assumption

  Access Paper or Ask Questions

How to Initialize your Network? Robust Initialization for WeightNorm & ResNets


Jun 05, 2019
Devansh Arpit, Victor Campos, Yoshua Bengio

Figure 1 for How to Initialize your Network? Robust Initialization for WeightNorm & ResNets
Figure 2 for How to Initialize your Network? Robust Initialization for WeightNorm & ResNets
Figure 3 for How to Initialize your Network? Robust Initialization for WeightNorm & ResNets
Figure 4 for How to Initialize your Network? Robust Initialization for WeightNorm & ResNets

  Access Paper or Ask Questions

The Benefits of Over-parameterization at Initialization in Deep ReLU Networks


Jan 11, 2019
Devansh Arpit, Yoshua Bengio

Figure 1 for The Benefits of Over-parameterization at Initialization in Deep ReLU Networks
Figure 2 for The Benefits of Over-parameterization at Initialization in Deep ReLU Networks
Figure 3 for The Benefits of Over-parameterization at Initialization in Deep ReLU Networks
Figure 4 for The Benefits of Over-parameterization at Initialization in Deep ReLU Networks

  Access Paper or Ask Questions

On the Spectral Bias of Neural Networks


Oct 17, 2018
Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred A. Hamprecht, Yoshua Bengio, Aaron Courville

Figure 1 for On the Spectral Bias of Neural Networks
Figure 2 for On the Spectral Bias of Neural Networks
Figure 3 for On the Spectral Bias of Neural Networks
Figure 4 for On the Spectral Bias of Neural Networks

  Access Paper or Ask Questions

Three Factors Influencing Minima in SGD


Sep 13, 2018
Stanisław Jastrzębski, Zachary Kenton, Devansh Arpit, Nicolas Ballas, Asja Fischer, Yoshua Bengio, Amos Storkey

Figure 1 for Three Factors Influencing Minima in SGD
Figure 2 for Three Factors Influencing Minima in SGD
Figure 3 for Three Factors Influencing Minima in SGD
Figure 4 for Three Factors Influencing Minima in SGD
* First two authors contributed equally. Short version accepted into ICLR workshop. Accepted to Artificial Neural Networks and Machine Learning, ICANN 2018 
  Access Paper or Ask Questions

A Walk with SGD


May 30, 2018
Chen Xing, Devansh Arpit, Christos Tsirigotis, Yoshua Bengio

Figure 1 for A Walk with SGD
Figure 2 for A Walk with SGD
Figure 3 for A Walk with SGD
Figure 4 for A Walk with SGD
* First two authors contributed equally 
  Access Paper or Ask Questions

Fraternal Dropout


Mar 28, 2018
Konrad Zolna, Devansh Arpit, Dendi Suhubdy, Yoshua Bengio

Figure 1 for Fraternal Dropout
Figure 2 for Fraternal Dropout
Figure 3 for Fraternal Dropout
Figure 4 for Fraternal Dropout
* Accepted to ICLR 2018. Extended appendix. Added official GitHub code for replication: https://github.com/kondiz/fraternal-dropout . Added references. Corrected typos 
  Access Paper or Ask Questions

Residual Connections Encourage Iterative Inference


Mar 08, 2018
Stanisław Jastrzębski, Devansh Arpit, Nicolas Ballas, Vikas Verma, Tong Che, Yoshua Bengio

Figure 1 for Residual Connections Encourage Iterative Inference
Figure 2 for Residual Connections Encourage Iterative Inference
Figure 3 for Residual Connections Encourage Iterative Inference
Figure 4 for Residual Connections Encourage Iterative Inference
* First two authors contributed equally. Published in ICLR 2018 
  Access Paper or Ask Questions

Variational Bi-LSTMs


Nov 15, 2017
Samira Shabanian, Devansh Arpit, Adam Trischler, Yoshua Bengio

Figure 1 for Variational Bi-LSTMs
Figure 2 for Variational Bi-LSTMs
Figure 3 for Variational Bi-LSTMs
Figure 4 for Variational Bi-LSTMs

  Access Paper or Ask Questions

On Optimality Conditions for Auto-Encoder Signal Recovery


Jul 13, 2017
Devansh Arpit, Yingbo Zhou, Hung Q. Ngo, Nils Napp, Venu Govindaraju

Figure 1 for On Optimality Conditions for Auto-Encoder Signal Recovery
Figure 2 for On Optimality Conditions for Auto-Encoder Signal Recovery
Figure 3 for On Optimality Conditions for Auto-Encoder Signal Recovery
Figure 4 for On Optimality Conditions for Auto-Encoder Signal Recovery

  Access Paper or Ask Questions

A Closer Look at Memorization in Deep Networks


Jul 01, 2017
Devansh Arpit, Stanisław Jastrzębski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S. Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, Simon Lacoste-Julien

Figure 1 for A Closer Look at Memorization in Deep Networks
Figure 2 for A Closer Look at Memorization in Deep Networks
Figure 3 for A Closer Look at Memorization in Deep Networks
Figure 4 for A Closer Look at Memorization in Deep Networks
* Appears in Proceedings of the 34th International Conference on Machine Learning (ICML 2017), Devansh Arpit, Stanis{\l}aw Jastrz\k{e}bski, Nicolas Ballas, and David Krueger contributed equally to this work 
  Access Paper or Ask Questions

Normalization Propagation: A Parametric Technique for Removing Internal Covariate Shift in Deep Networks


Jul 12, 2016
Devansh Arpit, Yingbo Zhou, Bhargava U. Kota, Venu Govindaraju

Figure 1 for Normalization Propagation: A Parametric Technique for Removing Internal Covariate Shift in Deep Networks
Figure 2 for Normalization Propagation: A Parametric Technique for Removing Internal Covariate Shift in Deep Networks
Figure 3 for Normalization Propagation: A Parametric Technique for Removing Internal Covariate Shift in Deep Networks
Figure 4 for Normalization Propagation: A Parametric Technique for Removing Internal Covariate Shift in Deep Networks
* 11 pages, ICML 2016, appendix added to the last version 
  Access Paper or Ask Questions

Why Regularized Auto-Encoders learn Sparse Representation?


Jun 17, 2016
Devansh Arpit, Yingbo Zhou, Hung Ngo, Venu Govindaraju

Figure 1 for Why Regularized Auto-Encoders learn Sparse Representation?
Figure 2 for Why Regularized Auto-Encoders learn Sparse Representation?
Figure 3 for Why Regularized Auto-Encoders learn Sparse Representation?
Figure 4 for Why Regularized Auto-Encoders learn Sparse Representation?
* 8 pages of content, 1 page of reference, 4 pages of supplementary. ICML 2016; bug fix in lemma 1 
  Access Paper or Ask Questions

Dimensionality Reduction with Subspace Structure Preservation


Apr 06, 2016
Devansh Arpit, Ifeoma Nwogu, Venu Govindaraju

Figure 1 for Dimensionality Reduction with Subspace Structure Preservation
Figure 2 for Dimensionality Reduction with Subspace Structure Preservation
Figure 3 for Dimensionality Reduction with Subspace Structure Preservation
* Published in NIPS 2014; v2: minor updates to the algorithm and added a few lines addressing application to large-scale/high-dimensional data 
  Access Paper or Ask Questions

Is Joint Training Better for Deep Auto-Encoders?


Jun 15, 2015
Yingbo Zhou, Devansh Arpit, Ifeoma Nwogu, Venu Govindaraju

Figure 1 for Is Joint Training Better for Deep Auto-Encoders?
Figure 2 for Is Joint Training Better for Deep Auto-Encoders?
Figure 3 for Is Joint Training Better for Deep Auto-Encoders?
Figure 4 for Is Joint Training Better for Deep Auto-Encoders?
* 11 pages, 4 figures 
  Access Paper or Ask Questions

An Analysis of Random Projections in Cancelable Biometrics


Nov 14, 2014
Devansh Arpit, Ifeoma Nwogu, Gaurav Srivastava, Venu Govindaraju

Figure 1 for An Analysis of Random Projections in Cancelable Biometrics
Figure 2 for An Analysis of Random Projections in Cancelable Biometrics
Figure 3 for An Analysis of Random Projections in Cancelable Biometrics
Figure 4 for An Analysis of Random Projections in Cancelable Biometrics

  Access Paper or Ask Questions