Get our free extension to see links to code for papers anywhere online!

Add to Chrome

Add to Firefox

Picture for Devansh Arpit

Devansh Arpit

Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization

Dec 28, 2020
Stanislaw Jastrzebski, Devansh Arpit, Oliver Astrand, Giancarlo Kerg, Huan Wang, Caiming Xiong, Richard Socher, Kyunghyun Cho, Krzysztof Geras

Figure 1 for Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization

Figure 2 for Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization

Figure 3 for Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization

Figure 4 for Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization

* The last two authors contributed equally

Access Paper or Ask Questions

The Break-Even Point on Optimization Trajectories of Deep Neural Networks

Feb 21, 2020
Stanislaw Jastrzebski, Maciej Szymczak, Stanislav Fort, Devansh Arpit, Jacek Tabor, Kyunghyun Cho, Krzysztof Geras

Figure 1 for The Break-Even Point on Optimization Trajectories of Deep Neural Networks

Figure 2 for The Break-Even Point on Optimization Trajectories of Deep Neural Networks

Figure 3 for The Break-Even Point on Optimization Trajectories of Deep Neural Networks

Figure 4 for The Break-Even Point on Optimization Trajectories of Deep Neural Networks

* Accepted as a spotlight at ICLR 2020. The last two authors contributed equally

Access Paper or Ask Questions

Entropy Penalty: Towards Generalization Beyond the IID Assumption

Oct 01, 2019
Devansh Arpit, Caiming Xiong, Richard Socher

Figure 1 for Entropy Penalty: Towards Generalization Beyond the IID Assumption

Figure 2 for Entropy Penalty: Towards Generalization Beyond the IID Assumption

Figure 3 for Entropy Penalty: Towards Generalization Beyond the IID Assumption

Figure 4 for Entropy Penalty: Towards Generalization Beyond the IID Assumption

Access Paper or Ask Questions

How to Initialize your Network? Robust Initialization for WeightNorm & ResNets

Jun 05, 2019
Devansh Arpit, Victor Campos, Yoshua Bengio

Figure 1 for How to Initialize your Network? Robust Initialization for WeightNorm & ResNets

Figure 2 for How to Initialize your Network? Robust Initialization for WeightNorm & ResNets

Figure 3 for How to Initialize your Network? Robust Initialization for WeightNorm & ResNets

Figure 4 for How to Initialize your Network? Robust Initialization for WeightNorm & ResNets

Access Paper or Ask Questions

The Benefits of Over-parameterization at Initialization in Deep ReLU Networks

Jan 11, 2019
Devansh Arpit, Yoshua Bengio

Figure 1 for The Benefits of Over-parameterization at Initialization in Deep ReLU Networks

Figure 2 for The Benefits of Over-parameterization at Initialization in Deep ReLU Networks

Figure 3 for The Benefits of Over-parameterization at Initialization in Deep ReLU Networks

Figure 4 for The Benefits of Over-parameterization at Initialization in Deep ReLU Networks

Access Paper or Ask Questions