Get our free extension to see links to code for papers anywhere online!

 Add to Chrome

 Add to Firefox

CatalyzeX Code Finder - Browser extension linking code for ML papers across the web! | Product Hunt Embed

How far can we go without convolution: Improving fully-connected networks

Nov 09, 2015
Zhouhan Lin, Roland Memisevic, Kishore Konda



We propose ways to improve the performance of fully connected networks. We found that two approaches in particular have a strong effect on performance: linear bottleneck layers and unsupervised pre-training using autoencoders without hidden unit biases. We show how both approaches can be related to improving gradient flow and reducing sparsity in the network. We show that a fully connected network can yield approximately 70% classification accuracy on the permutation-invariant CIFAR-10 task, which is much higher than the current state-of-the-art. By adding deformations to the training data, the fully connected network achieves 78% accuracy, which is just 10% short of a decent convolutional network.

* 10 pages, 11 figures, submitted for ICLR 2016 


Share this with someone who'll enjoy it:

   Access Paper Source



Share this with someone who'll enjoy it: