University of Oulu

L. Lu, Y. Jiang, M. Bennis, Z. Ding, F. Zheng and X. You, "Distributed Edge Caching via Reinforcement Learning in Fog Radio Access Networks," 2019 IEEE 89th Vehicular Technology Conference (VTC2019-Spring), Kuala Lumpur, Malaysia, 2019, pp. 1-6, https://doi.org/10.1109/VTCSpring.2019.8746321

Distributed edge caching via reinforcement learning in fog radio access networks

Saved in:
Author: Lu, Liuyang1,2,3; Jiang, Yanxiang1,2,3; Bennis, Mehdi4;
Organizations: 1National Mobile Communications Research Laboratory, Southeast University, Nanjing 210096, China
2State Key Laboratory of Integrated Services Networks, Xidian University, Xi’an 710071, China
3Key Laboratory of Wireless Sensor Network & Communication, Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences, 865 Changning Road, Shanghai 200050, China
4Centre for Wireless Communications, University of Oulu, Oulu 90014, Finland
5School of Electrical and Electronic Engineering, University of Manchester, Manchester, UK
6School of Electronic and Information Engineering, Harbin Institute of Technology, Shenzhen 518055, China
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 2.3 MB)
Persistent link: http://urn.fi/urn:nbn:fi-fe2020050424733
Language: English
Published: Institute of Electrical and Electronics Engineers, 2019
Publish Date: 2020-05-04
Description:

Abstract

In this paper, the distributed edge caching problem in fog radio access networks (F-RANs) is investigated. By considering the unknown spatio-temporal content popularity and user preference, a user request model based on hidden Markov process is proposed to characterize the fluctuant spatio-temporal traffic demands in F-RANs. Then, the Q-learning method based on the reinforcement learning (RL) framework is put forth to seek the optimal caching policy in a distributed manner, which enables fog access points (F-APs) to learn and track the potential dynamic process without extra communications cost. Furthermore, we propose a more efficient Q-learning method with value function approximation (Q-VFA-learning) to reduce complexity and accelerate convergence. Simulation results show that the performance of our proposed method is superior to those of the traditional methods.

see all

Series: IEEE Vehicular Technology Conference
ISSN: 1090-3038
ISSN-L: 1090-3038
ISBN: 978-1-7281-1217-6
ISBN Print: 978-1-7281-1218-3
Pages: 1 - 6
DOI: 10.1109/VTCSpring.2019.8746321
OADOI: https://oadoi.org/10.1109/VTCSpring.2019.8746321
Host publication: 2019 IEEE 89th Vehicular Technology Conference (VTC Spring). 28 April – 1 May 2019, Kuala Lumpur, Malaysia
Conference: IEEE Vehicular Technology Conference
Type of Publication: A4 Article in conference proceedings
Field of Science: 213 Electronic, automation and communications engineering, electronics
Subjects:
Funding: This work was supported in part by the Natural Science Foundation of China under Grant 61521061, the Natural Science Foundation of Jiangsu Province under grant BK20181264, the Research Fund of the State Key Laboratory of Integrated Services Networks (Xidian University) under grant ISN19-10, the Research Fund of the Key Laboratory of Wireless Sensor Network & Communication (Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences) under grant 2017002, the National Basic Research Program of China (973 Program)under grant 2012CB316004, and the U.K. Engineering and Physical Sciences Research Council under Grant EP/K040685/2.
Copyright information: © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.