University of Oulu

Q. Zhao, S. Paris, T. Veijalainen and S. Ali, "Hierarchical Multi-Objective Deep Reinforcement Learning for Packet Duplication in Multi-Connectivity for URLLC," 2021 Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit), 2021, pp. 142-147, doi: 10.1109/EuCNC/6GSummit51104.2021.9482453

Hierarchical multi-objective deep reinforcement learning for packet duplication in multi-connectivity for URLLC

Saved in:
Author: Zhao, Qiyang1; Paris, Stefano1; Veijalainen, Teemu2;
Organizations: 1Nokia Bell Labs, Paris, France
2Nokia Bell Labs, Espoo, Finland
3Nokia Bell Labs, Oulu, Finland
4University of Oulu, Oulu, Finland
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 0.5 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2021
Publish Date: 2021-10-21


In this paper, machine learning solutions have been investigated to improve the decision of packet duplication in a multi-connectivity cellular network to optimize the satisfaction of delay and reliability in 5G. A multi-agent deep reinforcement learning scheme with sequential actor-critic model has been developed to improve the decision of packet duplication from observations of radio environment including channel state, interference and load. A multi-objective reward function has been developed to minimize the transmission delay, error rate and maximize satisfaction of the URLLC targets. System-level simulations have been carried out in a heterogeneous network by utilizing dual connectivity between macro and small cells. Our deep reinforcement learning scheme is shown to prioritize packet duplication to the UE where it gains from lower queueing and interference. Comparing with standard 5G multi-connectivity, it reduces the overall packet error rate and delay, with increased satisfaction rate of URLLC targets. Furthermore, it improves the network throughput and resource efficiency in dynamic user traffic with lower redundancy.

see all

Series: European Conference on Networks and Communications
ISSN: 2475-6490
ISSN-E: 2575-4912
ISSN-L: 2475-6490
ISBN: 978-1-6654-1526-2
ISBN Print: 978-1-6654-3021-0
Pages: 142 - 147
DOI: 10.1109/EuCNC/6GSummit51104.2021.9482453
Host publication: 2021 Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit)
Conference: Joint European Conference on Networks and Communications & 6G Summit
Type of Publication: A4 Article in conference proceedings
Field of Science: 113 Computer and information sciences
Funding: This work was supported by the Academy of Finland 6Genesis Flagship (grant 318927), and in part by 5G-FORCE project.
Academy of Finland Grant Number: 318927
Detailed Information: 318927 (Academy of Finland Funding decision)
Copyright information: © 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.