Modern-day civil, mechanical, and aeronautical structures are transitioning towards a continuous, online, and automated maintenance paradigm in order to ensure increased safety and reliability. The field of structural health monitoring (SHM) is playing a key role in this respect and active sensing acousto-ultrasound guided-wave based SHM techniques have shown great promise due to their potential sensitivity to small changes in the structure. However, the methods' robustness and diagnosis capability become limited in the presence of environmental and operational variability such as changing temperature. In order to circumvent this difficulty, in this paper, a novel stochastic time series-based framework was adopted to model guided wave propagation under varying temperatures. Different stochastic time-varying time series models, such as Recursive Maximum Likelihood Time-varying Auto-Regressive (RML-TAR) and Recursive Maximum Likelihood Time-varying Auto-Regressive with Exogenous Excitation (RML-TARX) models are put forward to model and capture the underlying dynamics of guided wave propagation under varying temperatures. The steps and facets of the identification procedure are presented and clearly explained. Then the identified models are used to perform one-step-ahead prediction as well as "simulation" of the guided wave signals. In order to gain insight from a physics perspective, high-fidelity finite element (FE) models were also established to model the effect of temperature variation on guided wave propagation. Finally, surrogate models are formulated through the use of stochastic time-dependent RML-TARX models and compared with the FE models under varying temperatures.