University of Oulu

W. J. Yun et al., "SlimFL: Federated Learning With Superposition Coding Over Slimmable Neural Networks," in IEEE/ACM Transactions on Networking, doi: 10.1109/TNET.2022.3231864

SlimFL : federated learning with superposition coding over slimmable neural networks

Saved in:
Author: Yun, Won Joon1; Kwak, Yunseok1; Baek, Hankyul1;
Organizations: 1School of Electrical Engineering, Korea University, Seoul, South Korea
2Department of Electrical and Computer Engineering, Ajou University, Suwon, South Korea
3Department of Electrical and Computer Engineering, The University of Utah, Salt Lake City, UT, USA
4Centre for Wireless Communications, University of Oulu, Oulu, Finland
5School of Information Technology, Deakin University, Geelong, VIC, Australia
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 2.8 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2023
Publish Date: 2023-10-04


Federated learning (FL) is a key enabler for efficient communication and computing, leveraging devices’ distributed computing capabilities. However, applying FL in practice is challenging due to the local devices’ heterogeneous energy, wireless channel conditions, and non-independently and identically distributed (non-IID) data distributions. To cope with these issues, this paper proposes a novel learning framework by integrating FL and width-adjustable slimmable neural networks (SNN). Integrating FL with SNNs is challenging due to time-varying channel conditions and data distributions. In addition, existing multi-width SNN training algorithms are sensitive to the data distributions across devices, which makes SNN ill-suited for FL. Motivated by this, we propose a communication and energy-efficient SNN-based FL (named SlimFL ) that jointly utilizes superposition coding (SC) for global model aggregation and superposition training (ST) for updating local models. By applying SC, SlimFL exchanges the superposition of multiple-width configurations decoded as many times as possible for a given communication throughput. Leveraging ST, SlimFL aligns the forward propagation of different width configurations while avoiding inter-width interference during backpropagation. We formally prove the convergence of SlimFL. The result reveals that SlimFL is not only communication-efficient but also deals with non-IID data distributions and poor channel conditions, which is also corroborated by data-intensive simulations.

see all

Series: IEEE/ACM transactions on networking
ISSN: 1063-6692
ISSN-E: 1558-2566
ISSN-L: 1063-6692
Issue: Online first
DOI: 10.1109/tnet.2022.3231864
Type of Publication: A1 Journal article – refereed
Field of Science: 213 Electronic, automation and communications engineering, electronics
Funding: This work was supported in part by the National Research Foundation of Korea (NRF-Korea) under Grant 2022R1A2C2004869 and in part by the Institute of Information and Communications Technology Planning and Evaluation (IITP) Grant through the Korea Government [Ministry of Science and Information and Communications Technology (MSIT)], Intelligent 6G Wireless Access System, under Grant 2021-0-00467.
Copyright information: © 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.