University of Oulu

M. Hatami, M. Leinonen and M. Codreanu, "Online Caching Policy with User Preferences and Time-Dependent Requests: A Reinforcement Learning Approach," 2019 53rd Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, USA, 2019, pp. 1384-1388.

Online caching policy with user preferences and time-dependent requests : a reinforcement learning approach

Saved in:
Author: Hatami, Mohammad1; Leinonen, Markus1; Codreanu, Marian2
Organizations: 1Centre for Wireless Communications, University of Oulu, Finland
2Department of Science and Technology, Linköping University, Sweden
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 0.4 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2020
Publish Date: 2020-03-31


Content caching is a promising approach to reduce data traffic in the back-haul links. We consider a system where multiple users request items from a cache-enabled base station that is connected to a cloud. The users request items according to the user preferences in a time-dependent fashion, i.e., a user is likely to request the next chunk (item) of the file requested at a previous time slot. Whenever the requested item is not in the cache, the base station downloads it from the cloud and forwards it to the user. In the meanwhile, the base station decides whether to replace one item in the cache by the fetched item, or to discard it. We model the problem as a Markov decision process (MDP) and propose a novel state space that takes advantage of the dynamics of the users’ requests. We use reinforcement learning and propose a Q-learning algorithm to find an optimal cache replacement policy that maximizes the cache hit ratio without knowing the popularity profile distribution, probability distribution of items, and user preference model. Simulation results show that the proposed algorithm improves the cache hit ratio compared to other baseline policies.

see all

Series: Asilomar Conference on on Signals, Systems & Computers
ISSN: 2576-2303
ISSN-L: 2576-2303
ISBN: 978-1-7281-4300-2
ISBN Print: 978-1-7281-4301-9
Pages: 1384 - 1388
DOI: 10.1109/IEEECONF44664.2019.9048832
Host publication: 53rd Annual Asilomar Conference on Signals, Systems, and Computers 2019. Pasific Grove, USA, Nov 3-6, 2019
Host publication editor: Matthews, Michael B.
Conference: Annual Asilomar Conference on Signals, Systems, and Computers
Type of Publication: A4 Article in conference proceedings
Field of Science: 213 Electronic, automation and communications engineering, electronics
Funding: This research has been financially supported by the Infotech Oulu, the Academy of Finland (grant 323698), and Academy of Finland 6Genesis Flagship (grant 318927). M. Codreanu would like to acknowledge the support of the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie Grant Agreement No. 793402 (COMPRESS NETS).
Academy of Finland Grant Number: 323698
Detailed Information: 323698 (Academy of Finland Funding decision)
318927 (Academy of Finland Funding decision)
Copyright information: © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.