University of Oulu

Wang, Y.; Mäntylä, M.; Demeyer, S.; Wiklund, K.; Eldh, S. and Kairi, T. (2020). Software Test Automation Maturity: A Survey of the State of the Practice. In Proceedings of the 15th International Conference on Software Technologies - Volume 1: ICSOFT, ISBN 978-989-758-443-5, pages 27-38. DOI: 10.5220/0009766800270038

Software test automation maturity : a survey of the state of the practice

Saved in:
Author: Wang, Yuqing1; Mäntylä, Mika V.1; Demeyer, Serge2;
Organizations: 1M3S Research Unit, University of Oulu, Oulu, Finland
2Universiteit Antwerpen & Flanders Make, Antwerp, Belgium
3Ericsson AB, Stockholm, Sweden
4Eficode, Helsinki, Finland
Format: article
Version: published version
Access: open
Online Access: PDF Full Text (PDF, 2 MB)
Persistent link:
Language: English
Published: Science and Technology Publications, Lda, 2020
Publish Date: 2020-11-18


The software industry has seen an increasing interest in test automation. In this paper, we present a test automation maturity survey serving as a self-assessment for practitioners. Based on responses of 151 practitioners coming from above 101 organizations in 25 countries, we make observations regarding the state of the practice of test automation maturity: a) The level of test automation maturity in different organizations is differentiated by the practices they adopt; b) Practitioner reported the quite diverse situation with respect to different practices, e.g., 85% practitioners agreed that their test teams have enough test automation expertise and skills, while 47% of practitioners admitted that there is lack of guidelines on designing and executing automated tests; c) Some practices are strongly correlated and/or closely clustered; d) The percentage of automated test cases and the use of Agile and/or DevOps development models are good indicators for a higher test automation maturity level; (e) The roles of practitioners may affect response variation, e.g., QA engineers give the most optimistic answers, consultants give the most pessimistic answers. Our results give an insight into present test automation processes and practices and indicate chances for further improvement in the present industry.

see all

ISBN Print: 978-989-758-443-5
Pages: 27 - 38
DOI: 10.5220/0009766800270038
Host publication: 15th International Conference on Software Technologies, ICSOFT 2020
Host publication editor: van Sinderen, M.
Fill, H.-G.
Maciaszek, L.
Conference: International Conference on Software Technologies
Type of Publication: A4 Article in conference proceedings
Field of Science: 113 Computer and information sciences
Funding: This study is supported by TESTOMAT Project (ITEA3 ID number 16032), funded by Business Finland under Grant Decision ID 3192/31/2017.
Copyright information: © 2020 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved. CC BY-NC-ND 4.0.