University of Oulu

M. Shinzaki et al., "Zero-Shot Adaptation for mmWave Beam-Tracking on Overhead Messenger Wires Through Robust Adversarial Reinforcement Learning," in IEEE Transactions on Cognitive Communications and Networking, vol. 8, no. 1, pp. 232-245, March 2022, doi: 10.1109/TCCN.2021.3116231

Zero-shot adaptation for mmWave beam-tracking on overhead messenger wires through robust adversarial reinforcement learning

Saved in:
Author: Shinzaki, Masao1; Koda, Yusuke2; Yamamoto, Koji1;
Organizations: 1Graduate School of Informatics, Kyoto University, Kyoto 6068501, Japan
2Centre for Wireless Communications, University of Oulu, 90014 Oulu, Finland
3School of Engineering, Tokyo Institute of Technology, Tokyo 158-0084, Japan
4NTT Access Network Service Systems Laboratories, NTT Corporation, Yokosuka 2390847, Japan
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 7.6 MB)
Persistent link:
Language: English
Published: IEEE Communications Society, 2022
Publish Date: 2022-09-02


Millimeter wave (mmWave) beam-tracking based on machine learning enables the development of accurate tracking policies while obviating the need to periodically solve beam-optimization problems. However, its applicability is still arguable when training-test gaps exist in terms of environmental parameters that affect the node dynamics. From this skeptical point of view, the contribution of this study is twofold. First, by considering an example scenario, we confirm that the training-test gap adversely affects the beam-tracking performance. More specifically, we consider nodes placed on overhead messenger wires, where the node dynamics are affected by several environmental parameters, e.g., the wire mass and tension. Although these are particular scenarios, they yield insight into the validation of the training-test gap problems. Second, we demonstrate the feasibility of zero-shot adaptation as a solution, where a learning agent adapts to environmental parameters unseen during training. This is achieved by leveraging a robust adversarial reinforcement learning (RARL) technique, where such training-and-test gaps are regarded as disturbances by adversaries that are jointly trained with a legitimate beam-tracking agent. Numerical evaluations demonstrate that the beam-tracking policy learned via RARL can be applied to a wide range of environmental parameters without severely degrading the received power.

see all

Series: IEEE transactions on cognitive communications and networking
ISSN: 2372-2045
ISSN-E: 2332-7731
ISSN-L: 2372-2045
Volume: 8
Issue: 1
Pages: 232 - 245
DOI: 10.1109/TCCN.2021.3116231
Type of Publication: A1 Journal article – refereed
Field of Science: 213 Electronic, automation and communications engineering, electronics
Copyright information: © 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.