University of Oulu

X. Chen, C. Wu, H. Zhang, Y. Zhang, M. Bennis and H. Vuojala, "Decentralized Deep Reinforcement Learning for Delay-Power Tradeoff in Vehicular Communications," ICC 2019 - 2019 IEEE International Conference on Communications (ICC), Shanghai, China, 2019, pp. 1-6. doi: 10.1109/ICC.2019.8761949

Decentralized deep reinforcement learning for delay-power tradeoff in vehicular communications

Saved in:
Author: Chen, Xianfu1; Wu, Celimuge2; Zhang, Honggang3;
Organizations: 1VTT Technical Research Centre of Finland, Finland
2Graduate School of Informatics and Engineering, University of Electro- Communications, Japan
3College of Information Science and Electronic Engineering, Zhejiang University, China
4of Informatics, University of Oslo, Norway
5Centre for Wireless Communications, University of Oulu, Finland
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 0.4 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2019
Publish Date: 2020-02-19


This paper targets at the problem of radio resource management for expected long-term delay-power tradeoff in vehicular communications. At each decision epoch, the road side unit observes the global network state, allocates channels and schedules data packets for all vehicle user equipment-pairs (VUE-pairs). The decision-making procedure is modelled as a discrete-time Markov decision process (MDP). The technical challenges in solving an optimal control policy originate from highly spatial mobility of vehicles and temporal variations in data traffic. To simplify the decision-making process, we first decompose the MDP into a series of per-VUE-pair MDPs. We then propose an online long short-term memory based deep reinforcement learning algorithm to break the curse of high dimensionality in state space faced by each per-VUE-pair MDP. With the proposed algorithm, the optimal channel allocation and packet scheduling decision at each epoch can be made in a decentralized way in accordance with the partial observations of the global network state at the VUE-pairs. Numerical simulations validate the theoretical analysis and show the effectiveness of the proposed online learning algorithm.

see all

Series: IEEE International Conference on Communications
ISSN: 1550-3607
ISSN-E: 1938-1883
ISSN-L: 1550-3607
ISBN: 978-1-5386-8088-9
ISBN Print: 978-1-5386-8089-6
Pages: 1 - 6
DOI: 10.1109/ICC.2019.8761949
Host publication: ICC 2019 - 2019 IEEE International Conference on Communications (ICC)
Conference: IEEE International Conference on Communications
Type of Publication: A4 Article in conference proceedings
Field of Science: 213 Electronic, automation and communications engineering, electronics
Copyright information: © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.