Graph representation learning aims at transforming graph data into meaningful low-dimensional vectors to facilitate the employment of machine learning and data mining algorithms designed for general data. Most current graph representation learning approaches are transductive, which means that they require all the nodes in the graph are known when learning graph representations and these approaches cannot naturally generalize to unseen nodes. In this paper, we present a Fast Inductive Graph Representation Learning framework (FI-GRL) to learn nodes' low-dimensional representations. Our approach can obtain accurate representations for seen nodes with provable theoretical guarantees and can easily generalize to unseen nodes. Specifically, in order to explicitly decouple nodes' relations expressed by the graph, we transform nodes into a randomized subspace spanned by a random projection matrix. This stage is guaranteed to preserve the projection-cost of the normalized random walk matrix which is highly related to the normalized cut of the graph. Then feature extraction is achieved by conducting singular value decomposition on the obtained matrix sketch. By leveraging the property of projection-cost preservation on the matrix sketch, the obtained representation result is nearly optimal. To deal with unseen nodes, we utilize folding-in technique to learn their meaningful representations. Empirically, when the amount of seen nodes are larger than that of unseen nodes, FI-GRL always achieves excellent results. Our algorithm is fast, simple to implement and theoretically guaranteed. Extensive experiments on real datasets demonstrate the superiority of our algorithm on both efficacy and efficiency over both macroscopic level (clustering) and microscopic level (structural hole detection) applications.