Get our free extension to see links to code for papers anywhere online!

Chrome logo  Add to Chrome

Firefox logo Add to Firefox

Traditional and accelerated gradient descent for neural architecture search

Jun 26, 2020
Nicolas Garcia Trillos, Felix Morales, Javier Morales

Share this with someone who'll enjoy it:

In this paper, we introduce two algorithms for neural architecture search (NASGD and NASAGD) following the theoretical work by two of the authors [3]. Such work aims to introduce the conceptual basis for new notions of traditional and accelerated gradient descent algorithms for the optimization of a function on a semi-discrete space using ideas from optimal transport theory. Our methods, which use the network morphism framework introduced in [2], can analyze forty times as many architectures as the fastest methods in the literature [2,10] while using the same computational resources and time and achieving comparable levels of accuracy. For example, with NASGD, on CIFAR-10, our method designs and trains networks with an error rate of 4.06 in only 12 hours on a single GPU.

   Access Paper Source

Share this with someone who'll enjoy it: