R. Hashemi, S. Ali, E. M. Taghavi, N. H. Mahmood and M. Latva-Aho, "Deep Reinforcement Learning for Practical Phase Shift Optimization in RIS-assisted Networks over Short Packet Communications," 2022 Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit), 2022, pp. 518-523, doi: 10.1109/EuCNC/6GSummit54941.2022.9815804.
Deep reinforcement learning for practical phase shift optimization in RIS-assisted networks over short packet communications
|Author:||Hashemi, Ramin1; Ali, Samad1; Taghavi, Ehsan Moeen1;|
1Centre for Wireless Communications (CWC), University of Oulu, Oulu, Finland
|Online Access:||PDF Full Text (PDF, 0.7 MB)|
|Persistent link:|| http://urn.fi/urn:nbn:fi-fe2022082956575
Institute of Electrical and Electronics Engineers,
|Publish Date:|| 2022-08-29
We study the practical phase shift design in a non-ideal reconfigurable intelligent surface (RIS)-aided ultra-reliable and low-latency communication (URLLC) system under finite blocklength (FBL) regime by leveraging a novel deep reinforcement learning (DRL) algorithm named as twin-delayed deep deterministic policy gradient (TD3). First, assuming industrial automation system with multiple actuators, the signal-to-interference-plus-noise ratio (SINR) and achievable rate in FBL regime are identified for each actuator in terms of the phase shift configuration matrix at the RIS. The channel state information (CSI) variations due to feedback delay are also considered that result in channel coefficients’ obsolescence. Then, the problem framework is proposed where the objective is to maximize the total achievable FBL rate in all ACs, subject to the practical phase shift constraint at the RIS elements. Since the problem is intractable to solve using conventional optimization methods, we resort to employing an actor-critic policy gradient DRL algorithm based on TD3, which relies on interacting RIS with FA environment by taking actions which are the phase shifts at the RIS elements, to maximize the expected observed reward, which is defined as the total FBL rate. The numerical results show that optimizing the practical phase shifts in the RIS via the proposed TD3 method is highly beneficial to improve the network total FBL rate in comparison with typical DRL methods.
European Conference on Networks and Communications
2022 Joint European conference on networks and communications & 6G summit (EuCNC/6G Summit)
Joint European Conference on Networks and Communications & 6G Summit
|Type of Publication:||
A4 Article in conference proceedings
|Field of Science:||
213 Electronic, automation and communications engineering, electronics
This research has been supported by the Academy of Finland, 6G Flagship program under Grant 346208.
|Academy of Finland Grant Number:||
346208 (Academy of Finland Funding decision)
© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.