AoI minimization in status update control with energy harvesting sensors |
|
Author: | Hatami, Mohammad1; Leinonen, Markus1; Codreanu, Marian2 |
Organizations: |
1Centre for Wireless Communications—Radio Technologies, University of Oulu, Oulu, Finland 2Department of Science and Technology, Linkoping University, Sweden |
Format: | article |
Version: | accepted version |
Access: | open |
Online Access: | PDF Full Text (PDF, 1.6 MB) |
Persistent link: | http://urn.fi/urn:nbn:fi-fe202201209509 |
Language: | English |
Published: |
Institute of Electrical and Electronics Engineers,
2021
|
Publish Date: | 2022-01-20 |
Description: |
AbstractInformation freshness is crucial for time-critical IoT applications, e.g., monitoring and control. We consider an IoT status update system with users, energy harvesting sensors, and a cache-enabled edge node. The users receive time-sensitive information about physical quantities, each measured by a sensor. Users demand for the information from the edge node whose cache stores the most recently received measurements from each sensor. To serve a request, the edge node either commands the sensor to send an update or retrieves the aged measurement from the cache. We aim at finding the best actions of the edge node to minimize the average AoI of the served measurements at the users, termed on-demand AoI. We model this problem as a Markov decision process and develop reinforcement learning (RL) algorithms: model-based value iteration and model-free Q-learning. We also propose a Q-learning method for the realistic case where the edge node is informed about the sensors’ battery levels only via the status updates. The case under transmission limitations is also addressed. Furthermore, properties of an optimal policy are characterized. Simulation results show that an optimal policy is a threshold-based policy and that the proposed RL methods significantly reduce the average cost compared to several baselines. see all
|
Series: |
IEEE transactions on communications |
ISSN: | 0090-6778 |
ISSN-E: | 1558-0857 |
ISSN-L: | 0090-6778 |
Volume: | 69 |
Issue: | 12 |
Pages: | 8335 - 8351 |
DOI: | 10.1109/TCOMM.2021.3114681 |
OADOI: | https://oadoi.org/10.1109/TCOMM.2021.3114681 |
Type of Publication: |
A1 Journal article – refereed |
Field of Science: |
213 Electronic, automation and communications engineering, electronics |
Subjects: | |
Funding: |
This research has been financially supported in part by Infotech Oulu, the Academy of Finland (grant 323698), and Academy of Finland 6Genesis Flagship (grant 318927). Marian Codreanu would like to acknowledge the support of the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie Grant Agreement No. 793402 (COMPRESS NETS). The work of Markus Leinonen has also been financially supported in part by the Academy of Finland (grant 319485). Mohammad Hatami would like to acknowledge the support of HPY Research Foundation and Riitta ja Jorma J. Takanen Foundation. |
Academy of Finland Grant Number: |
323698 319485 318927 |
Detailed Information: |
323698 (Academy of Finland Funding decision) 319485 (Academy of Finland Funding decision) 318927 (Academy of Finland Funding decision) |
Copyright information: |
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. |