University of Oulu

R. Hashemi, S. Ali, E. M. Taghavi, N. H. Mahmood and M. Latva-Aho, "Deep Reinforcement Learning for Practical Phase Shift Optimization in RIS-assisted Networks over Short Packet Communications," 2022 Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit), 2022, pp. 518-523, doi: 10.1109/EuCNC/6GSummit54941.2022.9815804.

Deep reinforcement learning for practical phase shift optimization in RIS-assisted networks over short packet communications

Saved in:
Author: Hashemi, Ramin1; Ali, Samad1; Taghavi, Ehsan Moeen1;
Organizations: 1Centre for Wireless Communications (CWC), University of Oulu, Oulu, Finland
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 0.7 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2022
Publish Date: 2022-08-29


We study the practical phase shift design in a non-ideal reconfigurable intelligent surface (RIS)-aided ultra-reliable and low-latency communication (URLLC) system under finite blocklength (FBL) regime by leveraging a novel deep reinforcement learning (DRL) algorithm named as twin-delayed deep deterministic policy gradient (TD3). First, assuming industrial automation system with multiple actuators, the signal-to-interference-plus-noise ratio (SINR) and achievable rate in FBL regime are identified for each actuator in terms of the phase shift configuration matrix at the RIS. The channel state information (CSI) variations due to feedback delay are also considered that result in channel coefficients’ obsolescence. Then, the problem framework is proposed where the objective is to maximize the total achievable FBL rate in all ACs, subject to the practical phase shift constraint at the RIS elements. Since the problem is intractable to solve using conventional optimization methods, we resort to employing an actor-critic policy gradient DRL algorithm based on TD3, which relies on interacting RIS with FA environment by taking actions which are the phase shifts at the RIS elements, to maximize the expected observed reward, which is defined as the total FBL rate. The numerical results show that optimizing the practical phase shifts in the RIS via the proposed TD3 method is highly beneficial to improve the network total FBL rate in comparison with typical DRL methods.

see all

Series: European Conference on Networks and Communications
ISSN: 2475-6490
ISSN-E: 2575-4912
ISSN-L: 2475-6490
ISBN: 978-1-6654-9871-5
ISBN Print: 978-1-6654-9872-2
DOI: 10.1109/eucnc/6gsummit54941.2022.9815804
Host publication: 2022 Joint European conference on networks and communications & 6G summit (EuCNC/6G Summit)
Conference: 2022 Joint European conference on networks and communications & 6G summit (EuCNC/6G Summit)
Type of Publication: A4 Article in conference proceedings
Field of Science: 213 Electronic, automation and communications engineering, electronics
Funding: This research has been supported by the Academy of Finland, 6G Flagship program under Grant 346208.
Academy of Finland Grant Number: 346208
Detailed Information: 346208 (Academy of Finland Funding decision)
Copyright information: © 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.