Abstract

Unmanned aerial vehicles (UAVs) are the emerging vital components of millimeter wave (mmWave) wireless systems. Accurate beam alignment is essential for efficient beam based mmWave communications of UAVs with base stations (BSs). Conventional beam sweeping approaches often have large overhead due to the high mobility and autonomous operation of UAVs. Learning-based approaches greatly reduce the overhead by leveraging UAV data, like position to identify optimal beam directions. In this paper, we propose a deep Q-Network(DQN)-based framework for uplink UAV-BS beam alignment where the UAV hovers around 5G new radio (NR) BS coverage area, with varying channel conditions. The proposed learning framework uses the location information and maximize the beamforming gain upon every communication request from UAV inside the multi-location environment. We compare the proposed framework against multi-armed bandit (MAB)-based and exhaustive approaches, respectively and then analyse its training performance over different coverage area requirements, antenna configurations and channel conditions. Our results show that the proposed framework converge faster than the MAB-based approach and comparable to traditional exhaustive approach in an online manner under real-time conditions. Moreover, this approach can be further enhanced to predict the optimal beams for unvisited UAV locations inside the coverage using correlation from neighbouring grid locations.