Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Giancarlo Kerg

Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization


Dec 28, 2020
Stanislaw Jastrzebski, Devansh Arpit, Oliver Astrand, Giancarlo Kerg, Huan Wang, Caiming Xiong, Richard Socher, Kyunghyun Cho, Krzysztof Geras

* The last two authors contributed equally 

  Access Paper or Ask Questions

Advantages of biologically-inspired adaptive neural activation in RNNs during learning


Jun 22, 2020
Victor Geadah, Giancarlo Kerg, Stefan Horoi, Guy Wolf, Guillaume Lajoie


  Access Paper or Ask Questions

Untangling tradeoffs between recurrence and self-attention in neural networks


Jun 16, 2020
Giancarlo Kerg, Bhargav Kanuparthi, Anirudh Goyal, Kyle Goyette, Yoshua Bengio, Guillaume Lajoie


  Access Paper or Ask Questions

Non-normal Recurrent Neural Network (nnRNN): learning long time dependencies while improving expressivity with transient dynamics


May 28, 2019
Giancarlo Kerg, Kyle Goyette, Maximilian Puelma Touzel, Gauthier Gidel, Eugene Vorontsov, Yoshua Bengio, Guillaume Lajoie


  Access Paper or Ask Questions