Analogs are nearest neighbors of the state of a system. By using analogs and their successors in time, one is able to produce empirical forecasts. Several analog forecasting methods have been used in atmospheric applications and tested on well-known dynamical systems. Although efficient in practice, theoretical connections between analog methods and dynamical systems have been overlooked. Analog forecasting can be related to the real dynamical equations of the system of interest. This study investigates the properties of different analog forecasting strategies by taking local approximations of the system's dynamics. We find that analog forecasting performances are highly linked to the local Jacobian matrix of the flow map, and that analog forecasting combined with linear regression allows to capture projections of this Jacobian matrix. The proposed methodology allows to estimate analog forecasting errors, and to compare different analog methods. These results are derived analytically and tested numerically on two simple chaotic dynamical systems.