University of Oulu

C. Ben Issaid, A. Elgabli, J. Park, M. Bennis and M. Debbah, "Communication Efficient Decentralized Learning Over Bipartite Graphs," in IEEE Transactions on Wireless Communications, vol. 21, no. 6, pp. 4150-4167, June 2022, doi: 10.1109/TWC.2021.3126859.

Communication efficient decentralized learning over bipartite graphs

Saved in:
Author: Ben Issaid, Chaouki1; Elgabli, Anis1; Park, Jihong2;
Organizations: 1Centre of Wireless Communications, University of Oulu, 90014 Oulu, Finland
2School of Information Technology, Deakin University, Geelong, VIC 3220, Australia
3Technology Innovation Institute, 9639 Masdar City, Abu Dhabi, United Arab Emirates and CentraleSupélec, University Paris-Saclay, 91192 Gif-sur-Yvette, France
Format: article
Version: published version
Access: open
Online Access: PDF Full Text (PDF, 2.6 MB)
Persistent link: http://urn.fi/urn:nbn:fi-fe2022020717882
Language: English
Published: Institute of Electrical and Electronics Engineers, 2021
Publish Date: 2022-02-07
Description:

Abstract

In this paper, we propose a communication-efficiently decentralized machine learning framework that solves a consensus optimization problem defined over a network of inter-connected workers. The proposed algorithm, Censored and Quantized Generalized GADMM (CQ-GGADMM), leverages the worker grouping and decentralized learning ideas of Group Alternating Direction Method of Multipliers (GADMM), and pushes the frontier in communication efficiency by extending its applicability to generalized network topologies, while incorporating link censoring for negligible updates after quantization. We theoretically prove that CQ-GGADMM achieves the linear convergence rate when the local objective functions are strongly convex under some mild assumptions. Numerical simulations corroborate that CQ-GGADMM exhibits higher communication efficiency in terms of the number of communication rounds and transmit energy consumption without compromising the accuracy and convergence speed, compared to the censored decentralized ADMM, and the worker grouping method of GADMM.

see all

Series: IEEE transactions on wireless communications
ISSN: 1536-1276
ISSN-E: 1558-2248
ISSN-L: 1536-1276
Volume: 21
Issue: 6
Pages: 4150 - 4167
DOI: 10.1109/TWC.2021.3126859
OADOI: https://oadoi.org/10.1109/TWC.2021.3126859
Type of Publication: A1 Journal article – refereed
Field of Science: 213 Electronic, automation and communications engineering, electronics
Subjects:
Funding: This work is supported by Academy of Finland 6G Flagship (grant no. 318927) and project SMARTER, projects EU-ICT IntellIoT (grant no. 957218) and EUCHISTERA LearningEdge, and CONNECT, Infotech-NOOR, and NEGEIN.
Academy of Finland Grant Number: 318927
Detailed Information: 318927 (Academy of Finland Funding decision)
Copyright information: © 2021 IEEE. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
  https://creativecommons.org/licenses/by/4.0/