Deep Learning-Based Computer Vision for Beam Selection and Proactive Blockage Prediction

Millimeter-wave communication faces two critical challenges: propagation losses requiring costly narrow-beam alignment, and penetration losses causing link failures from blocked line-of-sight paths. We address propagation loss through a novel vision-aided beam selection framework that integrates RGB imagery with received power profiles for efficient transmitter identification and beam prediction. This framework achieves 98.96% top-5 beam prediction accuracy, surpassing current state-of-the-art methods by at least 6% across all metrics. We address penetration loss through a proactive blockage prediction framework using a modified object tracker with weighted centroid-based depth estimation. This represents the first analysis of simultaneous non-uniform mobility of both transmitters and obstacles. Evaluated on completely unseen data, this framework achieves over 98% accuracy in predicting blockages up to three frames ahead, establishing strong performance benchmarks.

Deep Learning based Computer-vision for Enhanced Beamforming

Meeting the high data rate demands of modern applications necessitates the utilization of high-frequency spectrum bands, including millimeter-wave and sub-terahertz bands. However, these frequencies require precise alignment of narrow communication beams between transmitters and receivers, typically resulting in significant beam training overhead. This paper introduces a novel end-to-end vision-aided beamforming framework that utilizes images to predict optimal beams while considering geometric adjustments to reduce overhead. Our model demonstrates robust adaptability to dynamic environments without relying on additional training data where the experimental results indicate a top-5 beam prediction accuracy of 98.96%, significantly surpassing current state-of-the-art solutions in vision-aided beamforming.

KORSAL: Key-point Detection based Online Real-Time Spatio-Temporal Action Localization

Real-time and online action localization in a video is a critical yet highly challenging problem. Accurate action localization requires the utilization of both temporal and spatial information. Recent attempts achieve this by using computationally intensive 3D CNN architectures or highly redundant two-stream architectures with optical flow, making them both unsuitable for real-time, online applications. To accomplish activity localization under highly challenging real-time constraints, we propose utilizing fast and efficient key-point based bounding box prediction to spatially localize actions. We then introduce a tube-linking algorithm that maintains the continuity of action tubes temporally in the presence of occlusions. Further, we eliminate the need for a two-stream architecture by combining temporal and spatial information into a cascaded input to a single network, allowing the network to learn from both types of information. Temporal information is efficiently extracted using a structural similarity index map as opposed to computationally intensive optical flow. Despite the simplicity of our approach, our lightweight end-to-end architecture achieves state-of-the-art frame-mAP of 74.7% on the challenging UCF101-24 dataset, demonstrating a performance gain of 6.4% over the previous best online methods. We also achieve state-of-the-art video-mAP results compared to both online and offline methods. Moreover, our model achieves a frame rate of 41.8 FPS, which is a 10.7% improvement over contemporary real-time methods.