Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Improving Neural Network Robustness through Neighborhood Preserving Layers

Jan 28, 2021
Bingyuan Liu, Christopher Malon, Lingzhou Xue, Erik Kruus

Robustness against adversarial attack in neural networks is an important research topic in the machine learning community. We observe one major source of vulnerability of neural nets is from overparameterized fully-connected layers. In this paper, we propose a new neighborhood preserving layer which can replace these fully connected layers to improve the network robustness. We demonstrate a novel neural network architecture which can incorporate such layers and also can be trained efficiently. We theoretically prove that our models are more robust against distortion because they effectively control the magnitude of gradients. Finally, we empirically show that our designed network architecture is more robust against state-of-art gradient descent based attacks, such as a PGD attack on the benchmark datasets MNIST and CIFAR10.

* An earlier short version of this paper without proof is presented in 25th International Conference on Pattern Recognition(ICPR), Manifold Learning from Euclid to Riemann workshop 

Share this with someone who'll enjoy it:

   Access Paper Source

Share this with someone who'll enjoy it: