University of Oulu

A. Elgabli, J. Park, A. S. Bedi, M. Bennis and V. Aggarwal, "Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning," ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, 2020, pp. 8876-8880, doi: 10.1109/ICASSP40776.2020.9054491

Q-GADMM : quantized group ADMM for communication efficient decentralized machine learning

Saved in:
Author: Elgabli, Anis1; Park, Jihong1; Bedi, Amrit S.2;
Organizations: 1Center of Wireless Communication, University of Oulu, Finland
2Department of Electrical Engineering, IIT Kanpur
3School of Industrial Engineering and the School of Electrical and Computer Engineering, Purdue University, USASchool of Industrial Engineering and the School of Electrical and Computer Engineering, Purdue University, USA
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 1.1 MB)
Persistent link: http://urn.fi/urn:nbn:fi-fe2020062946101
Language: English
Published: Institute of Electrical and Electronics Engineers, 2020
Publish Date: 2020-06-29
Description:

Abstract

In this paper, we propose a communication-efficient decen-tralized machine learning (ML) algorithm, coined quantized group ADMM (Q-GADMM). Every worker in Q-GADMM communicates only with two neighbors, and updates its model via the group alternating direct method of multiplier (GADMM), thereby ensuring fast convergence while reducing the number of communication rounds. Furthermore, each worker quantizes its model updates before transmissions, thereby decreasing the communication payload sizes. We prove that Q-GADMM converges to the optimal solution for convex loss functions, and numerically show that Q-GADMM yields 7x less communication cost while achieving almost the same accuracy and convergence speed compared to GADMM without quantization.

see all

Series: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing
ISSN: 1520-6149
ISSN-E: 2379-190X
ISSN-L: 1520-6149
ISBN: 978-1-5090-6631-5
ISBN Print: 978-1-5090-6632-2
Pages: 8876 - 8880
DOI: 10.1109/ICASSP40776.2020.9054491
OADOI: https://oadoi.org/10.1109/ICASSP40776.2020.9054491
Host publication: ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) : Proceedings
Conference: IEEE International Conference on Acoustics, Speech, and Signal Processing
Type of Publication: A4 Article in conference proceedings
Field of Science: 213 Electronic, automation and communications engineering, electronics
Subjects:
Copyright information: © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.