University of Oulu

Q. Zhang, W. Saad and M. Bennis, "Millimeter Wave Communications With an Intelligent Reflector: Performance Optimization and Distributional Reinforcement Learning," in IEEE Transactions on Wireless Communications, vol. 21, no. 3, pp. 1836-1850, March 2022, doi: 10.1109/TWC.2021.3107520

Millimeter wave communications with an intelligent reflector : performance optimization and distributional reinforcement learning

Saved in:
Author: Zhang, Qianqian1,2; Saad, Walid1,3; Bennis, Mehdi4
Organizations: 1Bradley Department of Electrical and Computer Engineering, Virginia Tech, Blacksburg, VA 24061 USA
2Hughes Network Systems, Germantown, MD 20876 USA
3Department of Computer Science and Engineering, Kyung Hee University, Suwon, South Korea
4Center for Wireless Communications, University of Oulu, 90570 Oulu, Finland
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 0.8 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2022
Publish Date: 2022-08-31


In this paper, a novel framework is proposed to optimize the downlink multi-user communication of a millimeter wave base station, which is assisted by a reconfigurable intelligent reflector (IR). In particular, a channel estimation approach is developed to measure the channel state information (CSI) in real-time. First, for a perfect CSI scenario, the precoding transmission of the BS and the reflection coefficient of the IR are jointly optimized, via an iterative approach, so as to maximize the sum of downlink rates towards multiple users. Next, in the imperfect CSI scenario, a distributional reinforcement learning (DRL) approach is proposed to learn the optimal IR reflection and maximize the expectation of downlink capacity. In order to model the transmission rate’s probability distribution, a learning algorithm, based on quantile regression (QR), is developed, and the proposed QR-DRL method is proved to converge to a stable distribution of downlink transmission rate. Simulation results show that, in the error-free CSI scenario, the proposed approach yields over 30% and 2-fold increase in the downlink sum-rate, compared with a fixed IR reflection scheme and direct transmission scheme, respectively. Simulation results also show that by deploying more IR elements, the downlink sum-rate can be significantly improved. However, as the number of IR components increases, more time is required for channel estimation, and the slope of increase in the IR-aided transmission rate will become smaller. Furthermore, under limited knowledge of CSI, simulation results show that the proposed QR-DRL method, which learns a full distribution of the downlink rate, yields a better prediction accuracy and improves the downlink rate by 10% for online deployments, compared with a Q-learning baseline.

see all

Series: IEEE transactions on wireless communications
ISSN: 1536-1276
ISSN-E: 1558-2248
ISSN-L: 1536-1276
Volume: 21
Issue: 3
Pages: 1836 - 1850
DOI: 10.1109/twc.2021.3107520
Type of Publication: A1 Journal article – refereed
Field of Science: 213 Electronic, automation and communications engineering, electronics
Funding: This work was supported by the U.S. National Science Foundation under Grant CNS-1526844 and Grant CNS-2030215.
Copyright information: © 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.