Alert button

Negative eigenvalues of the Hessian in deep neural networks

Add code
Bookmark button
Alert button
Feb 06, 2019
Guillaume Alain, Nicolas Le Roux, Pierre-Antoine Manzagol

Figure 1 for Negative eigenvalues of the Hessian in deep neural networks
Figure 2 for Negative eigenvalues of the Hessian in deep neural networks
Figure 3 for Negative eigenvalues of the Hessian in deep neural networks
Figure 4 for Negative eigenvalues of the Hessian in deep neural networks

Share this with someone who'll enjoy it:

Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon

The loss function of deep networks is known to be non-convex but the precise nature of this nonconvexity is still an active area of research. In this work, we study the loss landscape of deep networks through the eigendecompositions of their Hessian matrix. In particular, we examine how important the negative eigenvalues are and the benefits one can observe in handling them appropriately.

View paper onarxiv icon

Share this with someone who'll enjoy it:

Twitter IconFacebook IconLinkedin IconWhatsapp IconMessenger Icon