University of Oulu

X. Wang, C. Wang, X. Li, V. C. M. Leung and T. Taleb, "Federated Deep Reinforcement Learning for Internet of Things With Decentralized Cooperative Edge Caching," in IEEE Internet of Things Journal, vol. 7, no. 10, pp. 9441-9455, Oct. 2020, doi: 10.1109/JIOT.2020.2986803

Federated deep reinforcement learning for Internet of Things with decentralized cooperative edge caching

Saved in:
Author: Wang, Xiaofei1; Wang, Chenyang1; Li, Xiuhua2,3,4;
Organizations: 1College of Intelligence and Computing, Tianjin University, Tianjin, 300072 China
2State Key Laboratory of Power Transmission Equipment and System Security and New Technology, Chongqing University, Chongqing 401331, China
3School of Big Data & Software Engineering, Chongqing University, Chongqing, 401331 China
4Key Laboratory of Dependable Service Computing in Cyber Physical Society (Chongqing University), Ministry of Education, China
5College of Computer Science & Software Engineering, Shenzhen University, Shenzhen, 518060 China
6Electrical and Computer Engineering, The University of British Columbia, Vancouver, BC, V6T 1Z4 Canada
7Department of Communications and Networking, School of Electrical Engineering, Aalto University, 02150 Espoo, Finland
8Information Technology and Electrical Engineering, Oulu University, Pentti Kaiteran katu 1, 90570 Oulu, Finland
9Department of Computer and Information Security, Sejong University, 209 Neungdong-ro, Gunja-dong, Gwangjin-gu, Seoul, 05006 South Korea
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 1.8 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2020
Publish Date: 2020-11-17


Edge caching is an emerging technology for addressing massive content access in mobile networks to support rapidly growing Internet-of-Things (IoT) services and applications. However, most current optimization-based methods lack a self-adaptive ability in dynamic environments. To tackle these challenges, current learning-based approaches are generally proposed in a centralized way. However, network resources may be overconsumed during the training and data transmission process. To address the complex and dynamic control issues, we propose a federated deep-reinforcement-learning-based cooperative edge caching (FADE) framework. FADE enables base stations (BSs) to cooperatively learn a shared predictive model by considering the first-round training parameters of the BSs as the initial input of the local training, and then uploads near-optimal local parameters to the BSs to participate in the next round of global training. Furthermore, we prove the expectation convergence of FADE. Trace-driven simulation results demonstrate the effectiveness of the proposed FADE framework on reducing the performance loss and average delay, offloading backhaul traffic, and improving the hit rate.

see all

Series: IEEE internet of things journal
ISSN: 2372-2541
ISSN-E: 2327-4662
ISSN-L: 2327-4662
Volume: 7
Issue: 10
Pages: 9441 - 9455
Article number: 9062302
DOI: 10.1109/JIOT.2020.2986803
Type of Publication: A1 Journal article – refereed
Field of Science: 213 Electronic, automation and communications engineering, electronics
Funding: This work is supported in part by the National Key R & D Program of China through grant No. 2019YFB2101901, 2018YFC0809803, 2018YFF0214700 and 2018YFF0214706, China NSFC through grants 61702364, 61902044 and 61672117, China NSFC GD Joint Fund U1701263, and Chongqing Research Program of Basic Research and Frontier Technology through Grant No. cstc2019jcyj-msxmX0589, Chinese National Engineering Laboratory for Big Data System Computing Technology, Canadian NSERC, the European Union’s Horizon 2020 Research and Innovation Program through the MonB5G Project under Grant No. 871780, the Academy of Finland 6Genesis project under Grant No. 318927, and the Academy of Finland CSN project under Grant No. 311654.
Academy of Finland Grant Number: 318927
Detailed Information: 318927 (Academy of Finland Funding decision)
Copyright information: © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.