University of Oulu

M. Hatami, M. Leinonen and M. Codreanu, "AoI Minimization in Status Update Control With Energy Harvesting Sensors," in IEEE Transactions on Communications, vol. 69, no. 12, pp. 8335-8351, Dec. 2021, doi: 10.1109/TCOMM.2021.3114681

AoI minimization in status update control with energy harvesting sensors

Saved in:
Author: Hatami, Mohammad1; Leinonen, Markus1; Codreanu, Marian2
Organizations: 1Centre for Wireless Communications—Radio Technologies, University of Oulu, Oulu, Finland
2Department of Science and Technology, Linkoping University, Sweden
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 1.6 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2021
Publish Date: 2022-01-20


Information freshness is crucial for time-critical IoT applications, e.g., monitoring and control. We consider an IoT status update system with users, energy harvesting sensors, and a cache-enabled edge node. The users receive time-sensitive information about physical quantities, each measured by a sensor. Users demand for the information from the edge node whose cache stores the most recently received measurements from each sensor. To serve a request, the edge node either commands the sensor to send an update or retrieves the aged measurement from the cache. We aim at finding the best actions of the edge node to minimize the average AoI of the served measurements at the users, termed on-demand AoI. We model this problem as a Markov decision process and develop reinforcement learning (RL) algorithms: model-based value iteration and model-free Q-learning. We also propose a Q-learning method for the realistic case where the edge node is informed about the sensors’ battery levels only via the status updates. The case under transmission limitations is also addressed. Furthermore, properties of an optimal policy are characterized. Simulation results show that an optimal policy is a threshold-based policy and that the proposed RL methods significantly reduce the average cost compared to several baselines.

see all

Series: IEEE transactions on communications
ISSN: 0090-6778
ISSN-E: 1558-0857
ISSN-L: 0090-6778
Volume: 69
Issue: 12
Pages: 8335 - 8351
DOI: 10.1109/TCOMM.2021.3114681
Type of Publication: A1 Journal article – refereed
Field of Science: 213 Electronic, automation and communications engineering, electronics
Funding: This research has been financially supported in part by Infotech Oulu, the Academy of Finland (grant 323698), and Academy of Finland 6Genesis Flagship (grant 318927). Marian Codreanu would like to acknowledge the support of the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie Grant Agreement No. 793402 (COMPRESS NETS). The work of Markus Leinonen has also been financially supported in part by the Academy of Finland (grant 319485). Mohammad Hatami would like to acknowledge the support of HPY Research Foundation and Riitta ja Jorma J. Takanen Foundation.
Academy of Finland Grant Number: 323698
Detailed Information: 323698 (Academy of Finland Funding decision)
319485 (Academy of Finland Funding decision)
318927 (Academy of Finland Funding decision)
Copyright information: © 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.