Autonomous driving has been an active area of research and development, with various strategies being explored for decision-making in autonomous vehicles. Rule-based systems, decision trees, Markov decision processes, and Bayesian networks have been some of the popular methods used to tackle the complexities of traffic conditions and avoid collisions. However, with the emergence of deep learning, many researchers have turned towards CNN-based methods to improve the performance of collision avoidance. Despite the promising results achieved by some CNN-based methods, the failure to establish correlations between sequential images often leads to more collisions. In this paper, we propose a CNN-based method that overcomes the limitation by establishing feature correlations between regions in sequential images using variants of attention. Our method combines the advantages of CNN in capturing regional features with a bi-directional LSTM to enhance the relationship between different local areas. Additionally, we use an encoder to improve computational efficiency. Our method takes "Bird's Eye View" graphs generated from camera and LiDAR sensors as input, simulates the position (x, y) and head offset angle (Yaw) to generate future trajectories. Experiment results demonstrate that our proposed method outperforms existing vision-based strategies, achieving an average of only 3.7 collisions per 1000 miles of driving distance on the L5kit test set. This significantly improves the success rate of collision avoidance and provides a promising solution for autonomous driving.