In this paper we propose a novel observer-based method for anomaly detection in connected and automated vehicles (CAVs). The proposed method utilizes an augmented extended Kalman filter (AEKF) to smooth sensor readings of a CAV based on a nonlinear car-following motion model with time delay, where the leading vehicle's trajectory is used by the subject vehicle to detect sensor anomalies. We use the classic $\chi^2$ fault detector in conjunction with the proposed AEKF for anomaly detection. To make the proposed model more suitable for real-world applications, we consider a stochastic communication time delay in the car-following model. Our experiments conducted on real-world connected vehicle data indicate that the AEKF with $\chi^2$-detector can achieve a high anomaly detection performance.