Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Introducing Noise in Decentralized Training of Neural Networks

Sep 27, 2018
Linara Adilova, Nathalie Paul, Peter Schlicht

Share this with someone who'll enjoy it:

It has been shown that injecting noise into the neural network weights during the training process leads to a better generalization of the resulting model. Noise injection in the distributed setup is a straightforward technique and it represents a promising approach to improve the locally trained models. We investigate the effects of noise injection into the neural networks during a decentralized training process. We show both theoretically and empirically that noise injection has no positive effect in expectation on linear models, though. However for non-linear neural networks we empirically show that noise injection substantially improves model quality helping to reach a generalization ability of a local model close to the serial baseline.

* ECML PKDD 2018, Workshop DMLE 
* 13 pages 

   Access Paper Source

Share this with someone who'll enjoy it: