Piecewise Linear Multilayer Perceptrons and Dropout

We propose a new type of hidden layer for a multilayer perceptron, and demonstrate that it obtains the best reported performance for an MLP on the MNIST dataset.