University of Oulu

M. Shokrnezhad, T. Taleb and P. Dazzi, "Double Deep Q-Learning-based Path Selection and Service Placement for Latency-Sensitive Beyond 5G Applications," in IEEE Transactions on Mobile Computing, doi: 10.1109/TMC.2023.3301506

Double Deep Q-Learning-based path selection and service placement for latency-sensitive Beyond 5G applications

Saved in:
Author: Shokrnezhad, Masoud1; Taleb, Tarik1; Dazzi, Patrizio2
Organizations: 1Oulu University, Oulu, Finland
2University of Pisa, Pisa, Italy
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 9.1 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2023
Publish Date: 2023-09-19


Nowadays, as the need for capacity continues to grow, entirely novel services are emerging. A solid cloud-network integrated infrastructure is necessary to supply these services in a real-time responsive, and scalable way. Due to their diverse characteristics and limited capacity, communication and computing resources must be collaboratively managed to unleash their full potential. Although several innovative methods have been proposed to orchestrate the resources, most ignored network resources or relaxed the network as a simple graph, focusing only on cloud resources. This paper fills the gap by studying the joint problem of communication and computing resource allocation, dubbed CCRA, including function placement and assignment, traffic prioritization, and path selection considering capacity constraints and quality requirements, to minimize total cost. We formulate the problem as a non-linear programming model and propose two approaches, dubbed B&B-CCRA and WF-CCRA, based on the Branch & Bound and Water-Filling algorithms to solve it when the system is fully known. Then, for partially known systems, a Double Deep Q-Learning (DDQL) architecture is designed. Numerical simulations show that B&CCRA optimally solves the problem, whereas WF-CCRA delivers near-optimal solutions in a substantially shorter time. Furthermore, it is demonstrated that DDQL-CCRA obtains near-optimal solutions in the absence of request-specific information.

see all

Series: IEEE transactions on mobile computing
ISSN: 1536-1233
ISSN-E: 1558-0660
ISSN-L: 1536-1233
Issue: Early access
DOI: 10.1109/TMC.2023.3301506
Type of Publication: A1 Journal article – refereed
Field of Science: 113 Computer and information sciences
Funding: This research work is partially supported by the Business Finland 6Bridge 6Core project under Grant No. 8410/31/2022, the Academy of Finland IDEA-MILL project under Grant No. 352428, the European Union’s Horizon 2020 ICT Cloud Computing program under the ACCORDION project with grant agreement No. 871793, and the Academy of Finland 6G Flagship program under Grant No. 346208.
Academy of Finland Grant Number: 352428
Detailed Information: 352428 (Academy of Finland Funding decision)
346208 (Academy of Finland Funding decision)
Copyright information: This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see