Joint caching and computing service placement for edge-enabled IoT based on deep reinforcement learning
Chen, Yan; Sun, Yanjing; Yang, Bin; Taleb, Tarik (2022-04-20)
Y. Chen, Y. Sun, B. Yang and T. Taleb, "Joint Caching and Computing Service Placement for Edge-Enabled IoT Based on Deep Reinforcement Learning," in IEEE Internet of Things Journal, vol. 9, no. 19, pp. 19501-19514, 1 Oct.1, 2022, doi: 10.1109/JIOT.2022.3168869
© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
https://rightsstatements.org/vocab/InC/1.0/
https://urn.fi/URN:NBN:fi-fe2022090257038
Tiivistelmä
Abstract
By placing edge service functions in proximity to IoT facilities, edge computing can satisfy various IoT applications’ resource and latency requirements. Sensing-data-driven IoT applications are prevalent in IoT systems, and their task processing relies on sensing data from sensors. Therefore, to ensure the quality of service (QoS) of such applications in an edge-enabled IoT system, dedicated caching functions (CFs) are required to cache necessary sensing data. This paper considers an edge-enabled IoT system and investigates the joint caching and computing service placement (JCCSP) problem for sensing-data-driven IoT applications. Then, deep reinforcement learning (DRL) is exploited since it can adapt to a heterogeneous system with limited prior knowledge. In the proposed DRL-based approaches, a policy network based on the encoder-decoder model is constructed to address the issue of varying sizes of JCCSP states and actions caused by different numbers of CFs related to applications. Then, an on-policy REINFORCE-based method is adopted to train the policy network. After that, an off-policy training method based on the twin-delayed (TD) deep deterministic policy gradient (DDPG) is proposed to enhance the training efficiency and experience utilization. In the proposed DDPG-based method, a weight-averaged twin-Q-delayed (WATQD) algorithm is introduced to reduce the bias of Q-value estimation. Simulation results show that our proposed DRL-based JCCSP approaches can achieve converged performance that is significantly superior to benchmarks. Moreover, compared with the original TD method, the proposed WATQD method can significantly improve the training stability.
Kokoelmat
- Avoin saatavuus [32049]