University of Oulu

J. -H. Lee, J. Park, M. Bennis and Y. -C. Ko, "Integrating LEO Satellites and Multi-UAV Reinforcement Learning for Hybrid FSO/RF Non-Terrestrial Networks," in IEEE Transactions on Vehicular Technology, vol. 72, no. 3, pp. 3647-3662, March 2023, doi: 10.1109/TVT.2022.3220696

Integrating LEO satellites and multi-UAV reinforcement learning for hybrid FSO/RF non-terrestrial networks

Saved in:
Author: Lee, Ju-Hyung1,2; Park, Jihong3; Bennis, Mehdi4;
Organizations: 1Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, CA 90007 USA
2School of Electrical and Computer Engineering, Korea University, Seoul 02841, Korea
3School of Information Technology, Deakin University, Geelong, VIC 3220, Australia
4Centre for Wireless Communications, University of Oulu, 90014 Oulu, Finland
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 2.1 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2022
Publish Date: 2023-04-05


Integrating low-altitude earth orbit (LEO) satellites (SATs) and unmanned aerial vehicles (UAVs) within a non-terrestrial network (NTN), we investigate the problem of forwarding packets between two faraway ground terminals through SAT and UAV relays using either radio-frequency (RF) or free-space optical (FSO) link. Towards maximizing the communication efficiency, the associations with orbiting SATs and the trajectories of UAVs should be optimized, which is challenging due to the time-varying network topology and a huge number of possible control actions. To overcome the difficulty, we lift this problem to multi-agent deep reinforcement learning with a novel action dimensionality reduction technique. Simulation results corroborate that our proposed SAT-UAV integrated scheme achieves 1.99x higher end-to-end sum throughput compared to a benchmark scheme with fixed ground relays. While improving the throughput, our proposed scheme also aims to reduce the UAV control energy, yielding 2.25x higher energy efficiency than a baseline method only maximizing the throughput. Lastly, thanks to utilizing hybrid FSO/RF links, the proposed scheme achieves up to 62.56x higher peak throughput and 21.09x higher worst-case throughput than the cases utilizing either RF or FSO links, highlighting the importance of co-designing SAT-UAV associations, UAV trajectories, and hybrid FSO/RF links in beyond-5 G NTNs.

see all

Series: IEEE transactions on vehicular technology
ISSN: 0018-9545
ISSN-E: 1939-9359
ISSN-L: 0018-9545
Volume: 72
Issue: 3
Pages: 3647 - 3662
DOI: 10.1109/TVT.2022.3220696
Type of Publication: A1 Journal article – refereed
Field of Science: 213 Electronic, automation and communications engineering, electronics
Funding: This work was supported in part by the Institute of Information & Communications Technology Planning & Evaluation (IITP) Grant funded by the Korea government (MSIT) under Grant 2021-0-00260 and in part by Research on LEO Inter-Satellite Links.
Copyright information: © 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.