To better select the correct training sample and obtain the robust representation of the query sample, this paper proposes a discriminant-based sparse optimization learning model. This learning model integrates discriminant and sparsity together. Based on this model, we then propose a classifier called locality-based discriminant sparse representation (LDSR). Because discriminant can help to increase the difference of samples in different classes and to decrease the difference of samples within the same class, LDSR can obtain better sparse coefficients and constitute a better sparse representation for classification. In order to take advantages of kernel techniques, discriminant and sparsity, we further propose a nonlinear classifier called kernel locality-based discriminant sparse representation (KLDSR). Experiments on several well-known databases prove that the performance of LDSR and KLDSR is better than that of several state-of-the-art methods including deep learning based methods.