P. Susarla, Y. Deng, M. Juntti and O. Sílven, "Hierarchial-DQN Position-Aided Beamforming for Uplink mmWave Cellular-Connected UAVs," GLOBECOM 2022 - 2022 IEEE Global Communications Conference, Rio de Janeiro, Brazil, 2022, pp. 1308-1313, doi: 10.1109/GLOBECOM48099.2022.10001044.
Hierarchial-DQN position-aided beamforming for uplink mmWave cellular-connected UAVs
|Author:||Susarla, Praneeth1; Deng, Yansha2; Juntti, Markku1;|
1University of Oulu, Finland
2King’s College London, United Kingdom
|Online Access:||PDF Full Text (PDF, 0.8 MB)|
|Persistent link:|| http://urn.fi/urn:nbn:fi-fe2023032332984
Institute of Electrical and Electronics Engineers,
|Publish Date:|| 2023-03-23
Unmanned aerial vehicles (UAVs) are the vital components of sixth generation (6G) millimeter wave (mmWave) wireless networks. Fast and reliable beam alignment is essential for efficient beam-based mmWave communications between UAVs and the base stations (BSs). Learning-based approaches may greatly reduce the overhead by leveraging UAV data, such as position, to identify the optimal beam directions. In this paper, we propose a deep reinforcement learning (DRL)-based framework for UAV-BS beam alignment using the hierarchical deep Q-Network (hDQN) in a mmWave radio setting. We consider uplink communications where the UAV hovers around 5G new radio (NR) BS coverage area, with three dimensional (3D) beams under diverse channel conditions. A BS serves with learnt beam-pairs in an uplink manner upon every communication request from UAV inside the multi-location environment. Compared to our prior DQN-based method, the proposed hDQN framework uses the location information and the fixed spatial arrangement of the antenna elements to reduce the beam search complexity and maximize the data rates efficiently. The results show that our proposed hDQN-based framework converges faster than the DQN-based approach with an average overall training reduction of 43% and, is generic to multi-location environments across different uniform planar array (UPA) configurations and diverse channel conditions.
|Pages:||1308 - 1313|
GLOBECOM 2022 : 2022 IEEE Global Communications Conference
IEEE Global Communications Conference
|Type of Publication:||
A4 Article in conference proceedings
|Field of Science:||
213 Electronic, automation and communications engineering, electronics
The research was supported by 6G Flagship (Grant No.346208), Finland and the Engineering and Physical ResearchCouncil (EPSRC), U.K., under Grant EP/W004348/1.
|Academy of Finland Grant Number:||
346208 (Academy of Finland Funding decision)
© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.