Alert button
Picture for Alena Kopanicakova

Alena Kopanicakova

Alert button

Training of deep residual networks with stochastic MG/OPT

Aug 09, 2021
Cyrill von Planta, Alena Kopanicakova, Rolf Krause

Figure 1 for Training of deep residual networks with stochastic MG/OPT
Figure 2 for Training of deep residual networks with stochastic MG/OPT
Figure 3 for Training of deep residual networks with stochastic MG/OPT
Figure 4 for Training of deep residual networks with stochastic MG/OPT

We train deep residual networks with a stochastic variant of the nonlinear multigrid method MG/OPT. To build the multilevel hierarchy, we use the dynamical systems viewpoint specific to residual networks. We report significant speed-ups and additional robustness for training MNIST on deep residual networks. Our numerical experiments also indicate that multilevel training can be used as a pruning technique, as many of the auxiliary networks have accuracies comparable to the original network.

Viaarxiv icon