University of Oulu

J. -H. Ahn, M. Bennis and J. Kang, "Model Compression via Pattern Shared Sparsification in Analog Federated Learning Under Communication Constraints," in IEEE Transactions on Green Communications and Networking, vol. 7, no. 1, pp. 298-312, March 2023, doi: 10.1109/TGCN.2022.3186538

Model compression via pattern shared sparsification in analog federated learning under communication constraints

Saved in:
Author: Ahn, Jin-Hyun1; Bennis, Mehdi2; Kang, Joonhyuk3
Organizations: 1MGH/BWH Center for Advanced Medical Computing and Analysis, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston MA 02114 USA
2Centre for Wireless Communications, University of Oulu, Oulu 90014, Finland
3School of Electrical Engineering (EE), Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Korea
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 10.4 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2022
Publish Date: 2023-03-20


Recently, it has been shown that analog transmission based federated learning enables more efficient usage of communication resources compared to the conventional digital transmission. In this paper, we propose an effective model compression strategy enabling analog FL under constrained communication bandwidth. To this end, the proposed approach is based on pattern shared sparsification by setting the same sparsification pattern of parameter vectors uploaded by edge devices, as opposed to each edge device independently applying sparsification. In particular, we propose specific schemes for determining the sparsification pattern and characterize the convergence of analog FL leveraging these proposed sparsification strategies, by deriving a closed-form upper boun d of convergence rate and residual error. The closed-form expression allows to capture the effect of communication bandwidth and power budget to the performance of analog FL. In terms of convergence analysis, the model parameter obtained with the proposed schemes is proven to converge to the optimum of model parameter. Numerical results show that leveraging the proposed pattern shared sparsification consistently improves the performance of analog FL in various settings of system parameters. The improvement in performance is more significant under scarce communication bandwidth and limited transmit power budget.

see all

Series: IEEE transactions on green communications and networking
ISSN: 2473-2400
ISSN-E: 2473-2400
ISSN-L: 2473-2400
Volume: 7
Issue: 1
Pages: 298 - 312
DOI: 10.1109/TGCN.2022.3186538
Type of Publication: A1 Journal article – refereed
Field of Science: 213 Electronic, automation and communications engineering, electronics
Funding: This work was supported by the Ministry of Science and ICT (MSIT), South Korea, under the Information Technology Research Center (ITRC) Support Program supervised by the Institute of Information and Communications Technology Planning and Evaluation (IITP) under Grant IITP-2020-0-01787.
Copyright information: © 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.