University of Oulu

X. Chen et al., "Age of Information Aware Radio Resource Management in Vehicular Networks: A Proactive Deep Reinforcement Learning Perspective," in IEEE Transactions on Wireless Communications, vol. 19, no. 4, pp. 2268-2281, April 2020. doi: 10.1109/TWC.2019.2963667

Age of information-aware radio resource management in vehicular networks : a proactive deep reinforcement learning perspective

Saved in:
Author: Chen, Xianfu1; Wu, Celimuge2; Chen, Tao1;
Organizations: 1VTT Technical Research Centre of Finland Ltd, Finland
2Graduate School of Informatics and Engineering, University of Electro-Communications, Japan
3Department of Informatics, University of Oslo, Norway
4Department of Mathematical and Systems Engineering, Shizuoka University, Japan
5Centre for Wireless Communications, University of Oulu, Finland
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 6.8 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2020
Publish Date: 2020-02-03


In this paper, we investigate the problem of age of information (AoI)-aware radio resource management for expected long-term performance optimization in a Manhattan grid vehicle-to-vehicle network. With the observation of global network state at each scheduling slot, the roadside unit (RSU) allocates the frequency bands and schedules packet transmissions for all vehicle user equipment-pairs (VUE-pairs). We model the stochastic decision-making procedure as a discrete-time single-agent Markov decision process (MDP). The technical challenges in solving the optimal control policy originate from high spatial mobility and temporally varying traffic information arrivals of the VUE-pairs. To make the problem solving tractable, we first decompose the original MDP into a series of per-VUE-pair MDPs. Then we propose a proactive algorithm based on long short-term memory and deep reinforcement learning techniques to address the partial observability and the curse of high dimensionality in local network state space faced by each VUE-pair. With the proposed algorithm, the RSU makes the optimal frequency band allocation and packet scheduling decision at each scheduling slot in a decentralized way in accordance with the partial observations of the global network state at the VUE-pairs. Numerical experiments validate the theoretical analysis and demonstrate the significant performance improvements from the proposed algorithm.

see all

Series: IEEE transactions on wireless communications
ISSN: 1536-1276
ISSN-E: 1558-2248
ISSN-L: 1536-1276
Volume: 19
Issue: 4
Pages: 2268 - 2281
DOI: 10.1109/TWC.2019.2963667
Type of Publication: A1 Journal article – refereed
Field of Science: 213 Electronic, automation and communications engineering, electronics
Funding: This work was supported in part by the Academy of Finland under Grant 319759, Grant 319758, and Grant 289611, in part by the National Key Research and Development Program of China under Grant 2017YFB1301003, in part by the National Natural Science Foundation of China under Grant 61701439 and Grant 61731002, in part by the Zhejiang Key Research and Development Plan under Grant 2019C01002, in part by the Japan Society for the Promotion of Science (JSPS) KAKENHI under Grant 18KK0279, Grant 18K18036, and Grant 19H04092, and in part by the Telecommunications Advanced Foundation.
Academy of Finland Grant Number: 319758
Detailed Information: 319758 (Academy of Finland Funding decision)
289611 (Academy of Finland Funding decision)
Copyright information: © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.