An introductory Generalization of the standard SVMs loss and its applications to Shallow and Deep Neural Networks

We propose a new convex loss for SVMs, both for the binary classification and for the regression models. Therefore, we show the mathematical derivation of the dual problems and we experiment them with several small data-sets. The minimal dimension of those data-sets is due to the difficult scalability of the SVM method to bigger instances. This preliminary study should prove that using pattern correlations inside the loss function could enhance the generalisation performances. Coherently, results show that generalisation measures are never worse than the standard losses and several times they are better. In our opinion, it should be considered a careful study of this loss, coupled with shallow and deep neural networks. In fact, we present some novel results obtained with those architectures.