Attention-weighted federated deep reinforcement learning for device-to-device assisted heterogeneous collaborative edge caching |
|
Author: | Wang, Xiaofei1; Li, Ruibin1; Wang, Chenyang1; |
Organizations: |
1College of Intelligence and Computing, Tianjin University, Tianjin, 300072 China 2Key Laboratory of Dependable Service Computing in Cyber Physical Society (Chongqing University), Ministry of Education, China 3School of Big Data & Software Engineering, Chongqing University, Chongqing, 401331 China
4Department of Communications and Networking, School of Electrical Engineering, Aalto University, 02150 Espoo, Finland
5Information Technology and Electrical Engineering, Oulu University, 90570 Oulu, Finland 6Department of Computer and Information Security, Sejong University, Seoul, 05006 South Korea 7College of Computer Science & Software Engineering, Shenzhen University, Shenzhen, 518060 China 8Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, V6T 1Z4 Canada |
Format: | article |
Version: | accepted version |
Access: | open |
Online Access: | PDF Full Text (PDF, 13.5 MB) |
Persistent link: | http://urn.fi/urn:nbn:fi-fe202103298628 |
Language: | English |
Published: |
Institute of Electrical and Electronics Engineers,
2021
|
Publish Date: | 2021-03-29 |
Description: |
AbstractIn order to meet the growing demands for multimedia service access and release the pressure of the core network, edge caching and device-to-device (D2D) communication have been regarded as two promising techniques in next generation mobile networks and beyond. However, most existing related studies lack consideration of effective cooperation and adaptability to the dynamic network environments. In this article, based on the flexible trilateral cooperation among user equipment, edge base stations and a cloud server, we propose a D2D-assisted heterogeneous collaborative edge caching framework by jointly optimizing the node selection and cache replacement in mobile networks. We formulate the joint optimization problem as a Markov decision process, and use a deep Q-learning network to solve the long-term mixed integer linear programming problem. We further design an attention-weighted federated deep reinforcement learning (AWFDRL) model that uses federated learning to improve the training efficiency of the Q-learning network by considering the limited computing and storage capacity, and incorporates an attention mechanism to optimize the aggregation weights to avoid the imbalance of local model quality. We prove the convergence of the corresponding algorithm, and present simulation results to show the effectiveness of the proposed AWFDRL framework in reducing average delay of content access, improving hit rate and offloading traffic. see all
|
Series: |
IEEE journal on selected areas in communications |
ISSN: | 0733-8716 |
ISSN-E: | 1558-0008 |
ISSN-L: | 0733-8716 |
Volume: | 39 |
Issue: | 1 |
Pages: | 154 - 169 |
DOI: | 10.1109/JSAC.2020.3036946 |
OADOI: | https://oadoi.org/10.1109/JSAC.2020.3036946 |
Type of Publication: |
A1 Journal article – refereed |
Field of Science: |
213 Electronic, automation and communications engineering, electronics |
Subjects: | |
Funding: |
This work is supported in part by the National Key R & D Program of China through Grants No. 2019YFB2101901, 2018YFC0809803 and 2018YFF0214700, National NSFC through Grants No. 62072332, 61902044, 62002260, 61672117 and 62072060, Chongqing Research Program of Basic Research and Frontier Technology (Grant No. cstc2019jcyj-msxmX0589), Fundamental Research Funds for the Central Universities (Grant No. 2020CDJQY-A022), the Academy of Finland Project CSN - under Grant Agreement 311654 and the 6Genesis project under Grant No. 318927, Chinese National Engineering Laboratory for Big Data System Computing Technology at Shenzhen University, and Canadian Natural Sciences and Engineering Research Council. |
Academy of Finland Grant Number: |
311654 318927 |
Detailed Information: |
311654 (Academy of Finland Funding decision) 318927 (Academy of Finland Funding decision) |
Copyright information: |
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. |