Practitioner evaluations on software testing tools
Raulamo-Jurvanen, Päivi; Hosio, Simo; Mäntylä, Mika V. (2019-04-01)
Päivi Raulamo-Jurvanen, Simo Hosio, and Mika V. Mäntylä. 2019. Practitioner Evaluations on Software Testing Tools. In Proceedings of the Evaluation and Assessment on Software Engineering (EASE ’19). Association for Computing Machinery, New York, NY, USA, 57–66. DOI:https://doi.org/10.1145/3319008.3319018
© 2019 Copyright held by the owner/author(s). Publication rights licensed to ACM. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in 23rd Evaluation and Assessment in Software Engineering Conference, EASE 2019; Copenhagen; Denmark; 14- 17 April 2019, https://doi.org/10.1145/3319008.3319018.
https://rightsstatements.org/vocab/InC/1.0/
https://urn.fi/URN:NBN:fi-fe202001142020
Tiivistelmä
Abstract
In software engineering practice, evaluating and selecting the software testing tools that best fit the project at hand is an important and challenging task. In scientific studies of software engineering, practitioner evaluations and beliefs have recently gained interest, and some studies suggest that practitioners find beliefs of peers more credible than empirical evidence. To study how software practitioners evaluate testing tools, we applied online opinion surveys (n=89). We analyzed the reliability of the opinions utilizing Krippendorff’s alpha, intra-class correlation coefficient (ICC), and coefficients of variation (CV). Negative binomial regression was used to evaluate the effect of demographics. We find that opinions towards a specific tool can be conflicting. We show how increasing the number of respondents improves the reliability of the estimates measured with ICC. Our results indicate that on average, opinions from seven experts provide a moderate level of reliability. From demographics, we find that technical seniority leads to more negative evaluations. To improve the understanding, robustness, and impact of the findings, we need to conduct further studies by utilizing diverse sources and complementary methods.
Kokoelmat
- Avoin saatavuus [31657]