There are a large number of methods for solving under-determined linear inverse problem. Many of them have very high time complexity for large datasets. We propose a new method called Two-Stage Sparse Representation (TSSR) to tackle this problem. We decompose the representing space of signals into two parts, the measurement dictionary and the sparsifying basis. The dictionary is designed to approximate a sub-Gaussian distribution to exploit its concentration property. We apply sparse coding to the signals on the dictionary in the first stage, and obtain the training and testing coefficients respectively. Then we design the basis to approach an identity matrix in the second stage, to acquire the Restricted Isometry Property (RIP) and universality property. The testing coefficients are encoded over the basis and the final representing coefficients are obtained. We verify that the projection of testing coefficients onto the basis is a good approximation of the signal onto the representing space. Since the projection is conducted on a much sparser space, the runtime is greatly reduced. For concrete realization, we provide an instance for the proposed TSSR. Experiments on four biometrics databases show that TSSR is effective and efficient, comparing with several classical methods for solving linear inverse problem.