University of Oulu

P. Susarla, B. Gouda, Y. Deng, M. Juntti, O. Sílven and A. Tölli, "DQN-based Beamforming for Uplink mmWave Cellular-Connected UAVs," 2021 IEEE Global Communications Conference (GLOBECOM), 2021, pp. 1-6, doi: 10.1109/GLOBECOM46510.2021.968508

DQN-based beamforming for uplink mmWave cellular-connected UAVs

Saved in:
Author: Susarla, Praneeth1; Gouda, Bikshapathi1; Deng, Yansha2;
Organizations: 1University of Oulu, Finland
2King’s College London, United Kingdom
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 0.6 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2021
Publish Date: 2022-02-28


Unmanned aerial vehicles (UAVs) are the emerging vital components of millimeter wave (mmWave) wireless systems. Accurate beam alignment is essential for efficient beam based mmWave communications of UAVs with base stations (BSs). Conventional beam sweeping approaches often have large overhead due to the high mobility and autonomous operation of UAVs. Learning-based approaches greatly reduce the overhead by leveraging UAV data, like position to identify optimal beam directions. In this paper, we propose a reinforcement learning (RL)-based framework for UAV-BS beam alignment using deep Q-Network (DQN) in a mmWave setting. We consider uplink communications where the UAV hovers around 5G new radio (NR) BS coverage area, with varying channel conditions. The proposed learning framework uses the location information to maximize data rate through the optimal beam-pairs efficiently, upon every communication request from UAV inside the multi-location environment. We compare our proposed framework against Multi-Armed Bandit (MAB) learning-based approach and the traditional exhaustive approach, respectively and also analyse the training performance of DQN-based beam alignment over different coverage area requirements and channel conditions. Our results show that the proposed DQN-based beam alignment converge faster and generic for different environmental conditions. The framework can also learn optimal beam alignment comparable to the exhaustive approach in an online manner under real-time conditions.

see all

ISBN: 978-1-7281-8104-2
ISBN Print: 978-1-7281-8105-9
Pages: 1 - 6
DOI: 10.1109/GLOBECOM46510.2021.9685080
Host publication: 2021 IEEE Global Communications Conference (GLOBECOM) : Proceedings
Conference: IEEE Global Communications Conference
Type of Publication: A4 Article in conference proceedings
Field of Science: 213 Electronic, automation and communications engineering, electronics
Funding: The research was supported by 6G Flagship programme, Finland and the Engineering and Physical Research Council (EPSRC), U.K., under Grant EP/w004348/1.
Copyright information: © 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.