Alert button

Global Convergence of Sobolev Training for Overparametrized Neural Networks

Jun 14, 2020
Jorio Cocola, Paul Hand

Share this with someone who'll enjoy it:

Sobolev loss is used when training a network to approximate the values and derivatives of a target function at a prescribed set of input points. Recent works have demonstrated its successful applications in various tasks such as distillation or synthetic gradient prediction. In this work we prove that an overparametrized two-layer relu neural network trained on the Sobolev loss with gradient flow from random initialization can fit any given function values and any given directional derivatives, under a separation condition on the input data.

* Accepted for presentation at the 6th International Conference on Machine Learning, Optimization and Data science - LOD 2020  
View paper onarxiv icon

Share this with someone who'll enjoy it: