Abstract

Future 5G/6G wireless networks will be increasingly using millimeter waves (mmWaves), where fast and efficient beamforming is vital for providing continuous service to highly mobile devices in the presence of interference and signal attenuation, manifested by blockage. In this paper, we propose a novel and efficient method for mmWave beamforming in massive multiple-input multiple-output (MIMO) systems to achieve the aforementioned goals with low complexity in such scenarios. In doing so, we utilize deep reinforcement learning (DRL) to maximize the network’s energy efficiency subject to the quality of service (QoS) constraint for each user equipment (UE) and obtain its hybrid beamforming matrices. In doing so, we assume each UE is simultaneously associated with multiple access points (APs), i.e., simultaneous beamforming to/from multiple APs (coordinated multipoints) is needed for each UE. We also propose a low-complexity training algorithm, based on approximate message passing, which is well suited for the network edge. Besides, we develop a distributed scheme to reduce communications overhead via federated DRL. Extensive simulations show significant performance improvement over existing methods.