University of Oulu

P. Susarla, Y. Deng, M. Juntti and O. Sílven, "Hierarchial-DQN Position-Aided Beamforming for Uplink mmWave Cellular-Connected UAVs," GLOBECOM 2022 - 2022 IEEE Global Communications Conference, Rio de Janeiro, Brazil, 2022, pp. 1308-1313, doi: 10.1109/GLOBECOM48099.2022.10001044.

Hierarchial-DQN position-aided beamforming for uplink mmWave cellular-connected UAVs

Saved in:
Author: Susarla, Praneeth1; Deng, Yansha2; Juntti, Markku1;
Organizations: 1University of Oulu, Finland
2King’s College London, United Kingdom
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 0.8 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2022
Publish Date: 2023-03-23


Unmanned aerial vehicles (UAVs) are the vital components of sixth generation (6G) millimeter wave (mmWave) wireless networks. Fast and reliable beam alignment is essential for efficient beam-based mmWave communications between UAVs and the base stations (BSs). Learning-based approaches may greatly reduce the overhead by leveraging UAV data, such as position, to identify the optimal beam directions. In this paper, we propose a deep reinforcement learning (DRL)-based framework for UAV-BS beam alignment using the hierarchical deep Q-Network (hDQN) in a mmWave radio setting. We consider uplink communications where the UAV hovers around 5G new radio (NR) BS coverage area, with three dimensional (3D) beams under diverse channel conditions. A BS serves with learnt beam-pairs in an uplink manner upon every communication request from UAV inside the multi-location environment. Compared to our prior DQN-based method, the proposed hDQN framework uses the location information and the fixed spatial arrangement of the antenna elements to reduce the beam search complexity and maximize the data rates efficiently. The results show that our proposed hDQN-based framework converges faster than the DQN-based approach with an average overall training reduction of 43% and, is generic to multi-location environments across different uniform planar array (UPA) configurations and diverse channel conditions.

see all

ISBN: 978-1-6654-3541-3
ISBN Print: 978-1-6654-3540-6
Pages: 1308 - 1313
DOI: 10.1109/globecom48099.2022.10001044
Host publication: GLOBECOM 2022 : 2022 IEEE Global Communications Conference
Conference: IEEE Global Communications Conference
Type of Publication: A4 Article in conference proceedings
Field of Science: 213 Electronic, automation and communications engineering, electronics
Funding: The research was supported by 6G Flagship (Grant No.346208), Finland and the Engineering and Physical ResearchCouncil (EPSRC), U.K., under Grant EP/W004348/1.
Academy of Finland Grant Number: 346208
Detailed Information: 346208 (Academy of Finland Funding decision)
Copyright information: © 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.