University of Oulu

Rantala, L., Mäntylä, M. & Lenarduzzi, V. Keyword-labeled self-admitted technical debt and static code analysis have significant relationship but limited overlap. Software Qual J (2023).

Keyword-labeled self-admitted technical debt and static code analysis have significant relationship but limited overlap

Saved in:
Author: Rantala, Leevi1; Mäntylä, Mika2; Lenarduzzi, Valentina1
Organizations: 1M3S / ITEE, University of Oulu, P.O.B. 4500, Oulu, 90014, Finland
2Computer Science / Faculty of Science, University of Helsinki, Helsinki, Finland
Format: article
Version: published version
Access: open
Online Access: PDF Full Text (PDF, 1.3 MB)
Persistent link:
Language: English
Published: Springer Nature, 2023
Publish Date: 2023-11-17


Technical debt presents sub-optimal choices made in development, which are beneficial in the short term but not in the long run. Consciously admitted debt, which is marked with a keyword, e.g., TODO, is called keyword-labeled self-admitted technical debt (KL-SATD). KL-SATD can lead to adverse effects in software development, e.g., to a rise in complexity within the developed software. We investigated the relationship between KL-SATD from source code comments and reports from the highly popular industrial program analysis tool SonarQube. The goal was to find which SonarQube metrics and issues are related to KL-SATD introduction and removal and how many KL-SATD in the context of an issue addresses that issue. We performed a study with 33 software repositories. We analyzed the changes in SonarQube reports (sqale index, reliability and security remediation metrics, and SonarQube issues) and the relationship to KL-SATD addition and removal with mixed model analysis. We manually annotated a sample to investigate how many KL-SATD comments are in the context of SonarQube issues and how many address them directly. KL-SATD is associated with a reduction in code maintainability measured with SonarQube’s sqale index. KL-SATD removal is associated with an increase in code maintainability (sqale index) and reliability measured with SonarQube’s reliability remediation effort. The introduction and removal of KL-SATD have a predominantly relationship with code smells, and not with vulnerabilities and bugs. Manual annotation revealed that 36% of KL-SATD comments are in the context of a SonarQube issue, but only 15% of the comment address an issue. This means that despite of statistical relationship between KL-SATD comments and SonarQube reports there is a large set of KL-SATD comments that are in areas that Sonarqube reports as clean or free of maintainability issues. KL-SATD introduction and removal are connected mainly to code smells, connecting them to maintainability rather than reliability or security. This is reinforced by the relationship with the sqale index, as well as the dominance of code smells in SonarQube issues. Many KL-SATD issues have characteristics going beyond static analysis tools and require future studies extending the capabilities of the current tools. As KL-SATD comments and SonarQube reports appear to have limited overlap, it suggests that they are complementary and both are needed for getting a comprehensive view coverage of code maintainability. The study also presents rules violations developers should be aware of regarding KL-SATD introduction and removal.

see all

Series: Software quality journal
ISSN: 0963-9314
ISSN-E: 1573-1367
ISSN-L: 0963-9314
Issue: Latest articles
DOI: 10.1007/s11219-023-09655-z
Type of Publication: A1 Journal article – refereed
Field of Science: 113 Computer and information sciences
Funding: The authors have been supported by Infotech Oulu and Academy of Finland (grants 298020 and 328058).
Academy of Finland Grant Number: 328058
Detailed Information: 328058 (Academy of Finland Funding decision)
298020 (Academy of Finland Funding decision)
Copyright information: © The Author(s) 2023. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit