Get our free extension to see links to code for papers anywhere online!

 Add to Chrome

 Add to Firefox

CatalyzeX Code Finder - Browser extension linking code for ML papers across the web! | Product Hunt Embed
Gradient Starvation: A Learning Proclivity in Neural Networks

Nov 23, 2020
Mohammad Pezeshki, Sékou-Oumar Kaba, Yoshua Bengio, Aaron Courville, Doina Precup, Guillaume Lajoie


  Access Paper or Ask Questions

LEAD: Least-Action Dynamics for Min-Max Optimization

Oct 26, 2020
Reyhane Askari Hemmat, Amartya Mitra, Guillaume Lajoie, Ioannis Mitliagkas


  Access Paper or Ask Questions

Implicit Regularization in Deep Learning: A View from Function Space

Aug 03, 2020
Aristide Baratin, Thomas George, César Laurent, R Devon Hjelm, Guillaume Lajoie, Pascal Vincent, Simon Lacoste-Julien

* 24 pages. A preliminary version of this work has been presented at the NeurIPS 2019 Workshops on "Machine Learning with Guarantees" and "Science meets Engineering of Deep Learning" 

  Access Paper or Ask Questions

Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modules

Jun 30, 2020
Sarthak Mittal, Alex Lamb, Anirudh Goyal, Vikram Voleti, Murray Shanahan, Guillaume Lajoie, Michael Mozer, Yoshua Bengio

* ICML 2020 

  Access Paper or Ask Questions

On Lyapunov Exponents for RNNs: Understanding Information Propagation Using Dynamical Systems Tools

Jun 25, 2020
Ryan Vogt, Maximilian Puelma Touzel, Eli Shlizerman, Guillaume Lajoie

* Associated github repository: https://github.com/shlizee/lyapunov-hyperopt 

  Access Paper or Ask Questions

Advantages of biologically-inspired adaptive neural activation in RNNs during learning

Jun 22, 2020
Victor Geadah, Giancarlo Kerg, Stefan Horoi, Guy Wolf, Guillaume Lajoie


  Access Paper or Ask Questions

Untangling tradeoffs between recurrence and self-attention in neural networks

Jun 16, 2020
Giancarlo Kerg, Bhargav Kanuparthi, Anirudh Goyal, Kyle Goyette, Yoshua Bengio, Guillaume Lajoie


  Access Paper or Ask Questions

Internal representation dynamics and geometry in recurrent neural networks

Jan 14, 2020
Stefan Horoi, Guillaume Lajoie, Guy Wolf

* Presented as a poster at MAIS 2019: the Montreal AI Symposium, Montreal, Quebec, Canada, 2019 

  Access Paper or Ask Questions

Dimensionality compression and expansion in Deep Neural Networks

Jun 02, 2019
Stefano Recanatesi, Matthew Farrell, Madhu Advani, Timothy Moore, Guillaume Lajoie, Eric Shea-Brown

* Submitted to NeurIPS 2019. First two authors contributed equally 

  Access Paper or Ask Questions

Non-normal Recurrent Neural Network (nnRNN): learning long time dependencies while improving expressivity with transient dynamics

May 28, 2019
Giancarlo Kerg, Kyle Goyette, Maximilian Puelma Touzel, Gauthier Gidel, Eugene Vorontsov, Yoshua Bengio, Guillaume Lajoie


  Access Paper or Ask Questions