M. Krouka, A. Elgabli, C. B. Issaid and M. Bennis, "Communication-Efficient and Federated Multi-Agent Reinforcement Learning," in IEEE Transactions on Cognitive Communications and Networking, vol. 8, no. 1, pp. 311-320, March 2022, doi: 10.1109/TCCN.2021.3130993
Communication-efficient and federated multi-agent reinforcement learning
|Author:||Krouka, Mounssif1; Elgabli, Anis1; Issaid, Chaouki Ben1;|
1Centre of Wireless Communications, University of Oulu, 90014 Oulu, Finland
|Online Access:||PDF Full Text (PDF, 22.2 MB)|
|Persistent link:|| http://urn.fi/urn:nbn:fi-fe2022012811205
IEEE Communications Society,
|Publish Date:|| 2022-01-28
In this paper, we consider a distributed reinforcement learning setting where agents are communicating with a central entity in a shared environment to maximize a global reward. A main challenge in this setting is that the randomness of the wireless channel perturbs each agent’s model update while multiple agents’ updates may cause interference when communicating under limited bandwidth. To address this issue, we propose a novel distributed reinforcement learning algorithm based on the alternating direction method of multipliers (ADMM) and “over air aggregation” using analog transmission scheme, referred to as A-RLADMM. Our algorithm incorporates the wireless channel into the formulation of the ADMM method, which enables agents to transmit each element of their updated models over the same channel using analog communication. Numerical experiments on a multi-agent collaborative navigation task show that our proposed algorithm significantly outperforms the digital communication baseline of A-RLADMM (DRLADMM), the lazily aggregated policy gradient (RL-LAPG), as well as the analog and the digital communication versions of the vanilla FL, (A-FRL) and (D-FRL) respectively.
IEEE transactions on cognitive communications and networking
|Pages:||311 - 320|
|Type of Publication:||
A1 Journal article – refereed
|Field of Science:||
213 Electronic, automation and communications engineering, electronics
This work is supported by Academy of Finland 6G Flagship (grant no. 318927) and project SMARTER, projects EU-ICT IntellIoT and EUCHISTERA LearningEdge, and CONNECT, Infotech-NOOR, and NEGEIN.
|EU Grant Number:||
(957218) IntellIoT - Intelligent, distributed, human-centered and trustworthy IoT environments
|Academy of Finland Grant Number:||
318927 (Academy of Finland Funding decision)
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.