Millimeter-wave transceivers use large antenna arrays to form narrow high-directional beams and overcome severe attenuation. Narrow beams require large signaling overhead to be aligned if no prior information about beam directions is available. Moreover, beams drift with time due to user mobility and may need to be realigned. Beam tracking is commonly used to keep the beams tightly coupled and eliminate the overhead associated with realignment. Hence, with periodic measurements, beams are adjusted before they lose alignment. We propose a model where the receiver adjusts beam direction "continuously" over each physical-layer sample according to a carefully calculated estimate of the continuous variation of the beams. In our approach, the change of direction is updated using the estimate of the variation rate of beam angles via two different methods, a Continuous-Discrete Kalman filter and an MMSE of a first-order approximation of the variation. Our approach incurs no additional overhead in pilots, yet, the performance of beam tracking is improved significantly. Numerical results reveal an SNR enhancement associated with reducing the MSE of the beam directions. In addition, our approach reduces the pilot overhead by 60% and up to 87% while achieving a similar total tracking duration as the state-of-the-art.