University of Oulu

S. Martínez-Fernández et al., "Continuously Assessing and Improving Software Quality With Software Analytics Tools: A Case Study," in IEEE Access, vol. 7, pp. 68219-68239, 2019. doi: 10.1109/ACCESS.2019.2917403

Continuously assessing and improving software quality with software analytics tools : a case study

Saved in:
Author: Martínez-Fernández, Silverio1; Vollmer, Anna Maria1; Jedlitschka, Andreas1;
Organizations: 1Fraunhofer Institute for Experimental Software Engineering (IESE), Kaiserslautern, Germany
2Department of Service and Information System Engineering, UPC-BarcelonaTech, Barcelona, Spain
3Faculty of Information Technology and Electrical Engineering, University of Oulu, Oulu, Finland
4Nokia, Oulu, Finland
5Softeam, Paris, France
6ITTI, Poznań, Poland
7Bittium, Oulu, Finland
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 1.3 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2019
Publish Date: 2019-09-23


In the last decade, modern data analytics technologies have enabled the creation of software analytics tools offering real-time visualization of various aspects related to software development and usage. These tools seem to be particularly attractive for companies doing agile software development. However, the information provided by the available tools is neither aggregated nor connected to higher quality goals. At the same time, assessing and improving the software quality has also been the key targets for the software engineering community, yielding several proposals for standards and software quality models. Integrating such quality models into software analytics tools could close the gap by providing the connection to higher quality goals. This paper aims at understanding whether the integration of quality models into software analytics tools provides understandable, reliable, useful, and relevant information at the right level of detail about the quality of a process or product and whether practitioners intend to use it. Over the course of more than a year, four companies involved in this case study deployed such a tool to assess and improve software quality in several projects. We used standardized measurement instruments to elicit the perception of 22 practitioners regarding their use of the tool. We complemented the findings with debriefing sessions held at the companies. In addition, we discussed challenges and lessons learned with four practitioners leading the use of the tool. The quantitative and qualitative analyses provided positive results, i.e., the practitioners’ perception with regard to the tool’s understandability, reliability, usefulness, and relevance was positive. Individual statements support the statistical findings, and constructive feedback can be used for future improvements. We conclude that the potential for future adoption of quality models within software analytics tools definitely exists and encourage other practitioners to use the presented seven challenges and seven lessons learned and adopt them in their companies.

see all

Series: IEEE access
ISSN: 2169-3536
ISSN-E: 2169-3536
ISSN-L: 2169-3536
Volume: 7
Pages: 68219 - 68239
DOI: 10.1109/ACCESS.2019.2917403
Type of Publication: A1 Journal article – refereed
Field of Science: 113 Computer and information sciences
Copyright information: © 2019 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission. See for more information.