University of Oulu

C. ben Issaid, A. Elgabli and M. Bennis, "Local Stochastic ADMM for Communication-Efficient Distributed Learning," 2022 IEEE Wireless Communications and Networking Conference (WCNC), Austin, TX, USA, 2022, pp. 1880-1885, doi: 10.1109/WCNC51071.2022.9771559

Local stochastic ADMM for communication-efficient distributed learning

Saved in:
Author: Ben Issaid, Chaouki1; Elgabli, Anis1; Bennis, Mehdi1
Organizations: 1Centre for Wireless Communications (CWC), University of Oulu, Finland
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 1.1 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2022
Publish Date: 2023-02-09


In this paper, we propose a communication-efficient alternating direction method of multipliers (ADMM)-based algorithm for solving a distributed learning problem in the stochastic non-convex setting. Our approach runs a few stochastic gradient descent (SGD) steps to solve the local problem at each worker instead of finding the exact/approximate solution as proposed by existing ADMM-based works. By doing so, the proposed framework strikes a good balance between the computation and communication costs. Extensive simulation results show that our algorithm significantly outperforms existing stochastic ADMM in terms of communication-efficiency, notably in the presence of non-independent and identically distributed (non-IID) data.

see all

Series: IEEE Wireless Communications and Networking Conference
ISSN: 1525-3511
ISSN-E: 1558-2612
ISSN-L: 1525-3511
ISBN: 978-1-6654-4266-4
ISBN Print: 978-1-6654-4267-1
Article number: 9771559
DOI: 10.1109/wcnc51071.2022.9771559
Conference: IEEE Wireless Communications and Networking Conference
Type of Publication: A4 Article in conference proceedings
Field of Science: 213 Electronic, automation and communications engineering, electronics
Copyright information: © 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.