In this paper, we propose a learning-based low-overhead beam alignment method for vehicle-to-infrastructure communication in vehicular networks. The main idea is to remotely infer the optimal beam directions at a target base station in future time slots, based on the CSI of a source base station in previous time slots. The proposed scheme can reduce channel acquisition and beam training overhead by replacing pilot-aided beam training with online inference from a sequence-to-sequence neural network. Simulation results based on ray-tracing channel data show that our proposed scheme achieves a $8.86\%$ improvement over location-based beamforming schemes with a positioning error of $1$m, and is within a $4.93\%$ performance loss compared with the genie-aided optimal beamformer.