University of Oulu

H. Baek et al., "Joint Superposition Coding and Training for Federated Learning over Multi-Width Neural Networks," IEEE INFOCOM 2022 - IEEE Conference on Computer Communications, London, United Kingdom, 2022, pp. 1729-1738, doi: 10.1109/INFOCOM48880.2022.9796733

Joint superposition coding and training for federated learning over multi-width neural networks

Saved in:
Author: Baek, Hankyul1; Yun, Won Joon1; Kwak, Yunseok1;
Organizations: 1Department of Electrical and Computer Engineering, Korea University, Seoul, Republic of Korea
2School of Software, Hallym University, Chuncheon, Republic of Korea
3Department of Electrical and Computer Engineering, University of Utah, Salt Lake City, UT, USA
4Centre for Wireless Communications, University of Oulu, Oulu, Finland
5School of Information Technology, Deakin University, Geelong, Australia
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 4.6 MB)
Persistent link:
Language: English
Published: IEEE, 2022
Publish Date: 2023-02-10


This paper aims to integrate two synergetic technologies, federated learning (FL) and width-adjustable slimmable neural network (SNN) architectures. FL preserves data privacy by exchanging the locally trained models of mobile devices. By adopting SNNs as local models, FL can flexibly cope with the time-varying energy capacities of mobile devices. Combining FL and SNNs is however non-trivial, particularly under wireless connections with time-varying channel conditions. Furthermore, existing multi-width SNN training algorithms are sensitive to the data distributions across devices, so are ill-suited to FL. Motivated by this, we propose a communication and energy efficient SNN-based FL (named SlimFL) that jointly utilizes superposition coding (SC) for global model aggregation and superposition training (ST) for updating local models. By applying SC, SlimFL exchanges the superposition of multiple width configurations that are decoded as many as possible for a given communication throughput. Leveraging ST, SlimFL aligns the forward propagation of different width configurations, while avoiding the inter-width interference during back propagation. We formally prove the convergence of SlimFL. The result reveals that SlimFL is not only communication-efficient but also can counteract non-IID data distributions and poor channel conditions, which is also corroborated by simulations.

see all

ISBN: 978-1-6654-5822-1
ISBN Print: 978-1-6654-5823-8
Pages: 1729 - 1738
DOI: 10.1109/infocom48880.2022.9796733
Host publication: IEEE INFOCOM 2022 - IEEE Conference on Computer Communications
Conference: IEEE Conference on Computer Communications
Type of Publication: A4 Article in conference proceedings
Field of Science: 213 Electronic, automation and communications engineering, electronics
Funding: This research was funded by IITP 2021-0-00467.
Copyright information: © 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.