A Multi-parameter Updating Fourier Online Gradient Descent Algorithm for Large-scale Nonlinear Classification

Large scale nonlinear classification is a challenging task in the field of support vector machine. Online random Fourier feature map algorithms are very important methods for dealing with large scale nonlinear classification problems. The main shortcomings of these methods are as follows: (1) Since only the hyperplane vector is updated during learning while the random directions are fixed, there is no guarantee that these online methods can adapt to the change of data distribution when the data is coming one by one. (2) The dimension of the random direction is often higher for obtaining better classification accuracy, which results in longer test time. In order to overcome these shortcomings, a multi-parameter updating Fourier online gradient descent algorithm (MPU-FOGD) is proposed for large-scale nonlinear classification problems based on a novel random feature map. In the proposed method, the suggested random feature map has lower dimension while the multi-parameter updating strategy can guarantee the learning model can better adapt to the change of data distribution when the data is coming one by one. Theoretically, it is proved that compared with the existing random Fourier feature maps, the proposed random feature map can give a tighter error bound. Empirical studies on several benchmark data sets demonstrate that compared with the state-of-the-art online random Fourier feature map methods, the proposed MPU-FOGD can obtain better test accuracy.