Visual-Inertial odometry is known to suffer from drifting especially over long-term runs. In this paper, we present GVINS, a non-linear optimization based system that tightly fuses GNSS raw measurements with visual and inertial information for real-time and drift-free state estimation. The proposed system combines merits from VIO and GNSS system, thus is able to achieve both local smoothness and global consistency. To associate global measurements with local states, a coarse-to-fine initialization procedure is proposed to efficiently online calibrate the transformation and initialize GNSS states from only a short window of measurements. The GNSS pseudorange and Doppler shift measurements are modelled and optimized under a factor graph framework along with visual and inertial constraints. For complex and GNSS-unfriendly area, the degenerate cases are discussed and carefully handled to ensure robustness. The engineering challenges involved in the system are also included to facilitate relevant GNSS fusion researches. Thanks to the tightly-coupled multi-sensor approach and system design, our estimator is able to recover the position and orientation in the global Earth frame, even with less than 4 satellites being tracked. We extensively evaluate the proposed system on simulation and real-world experiments, and the result demonstrates that our system substantially eliminates the drift of VIO and preserves the accuracy in spite of noisy GNSS measurements.