University of Oulu

Students versus professionals as experiment subjects : an investigation on the effectiveness of TDD on code quality

Saved in:
Author: Salman, Iflaah1
Organizations: 1University of Oulu, Faculty of Information Technology and Electrical Engineering, Department of Information Processing Science, Information Processing Science
Format: ebook
Version: published version
Access: open
Online Access: PDF Full Text (PDF, 6.5 MB)
Pages: 106
Persistent link:
Language: English
Published: Oulu : I. Salman, 2014
Publish Date: 2015-04-10
Thesis type: Master's thesis
Tutor: Tosun Misirli, Ayse
Reviewer: Tosun Misirli, Ayse
Juristo, Natalia


Background: Most of the software engineering empirical studies use students as subjects for conducting experiments. This raises the concern that whether results acquired through students are applicable to the industry or not. The researchers argue that experiment with students lack realism. This situation not only causes the threats to the generalizability of the research results but also becomes a potential barrier to the adoption of novice research in industry.

Aim: The objective of the study is to investigate, whether students are representative of professionals in software engineering experiments. We are addressing the objective by investigating the difference in the code quality of the two subject groups — students and professionals — in the context of the effectiveness of Test Driven Development (TDD) on the internal code quality. The study tests two hypotheses; the first one aims to test the difference in the code quality of the two subject groups. The second one aims to test the difference in the code quality of tasks implemented following TDD and Test-Last Development (TLD) methodologies.

Method: The study follows a quantitative research approach with the experimentation methodology. The study involved graduate students from academia and professionals from the industry in their respective environmental settings. Both of the subject groups implemented tasks in TDD and TLD approaches. The treatments of the first hypothesis are professionals and students for studying the difference in the two subject groups. Furthermore, TDD and TLD are the treatments of the second hypothesis for assessing the difference in the code quality of the two methodologies.

Results: Students differed professionals when compared for TLD and TDD1 implementations. The results couldn’t show the difference for TDD2 task. The code quality differed as an effect of TDD for professionals in the cases of TLD vs. TDD1 and TLD vs. TDD2. Students’ data showed the difference in the code quality for TLD vs. TDD1 but couldn’t for TLD vs. TDD2 tasks. The null hypotheses were refuted with the p-value < 0.05.

Conclusion: The results of the study are confined to the context of TDD and to some extent with the experimental design. The future extension can be to compare the two subject groups by designing the same experiment. Additionally, comparison can also be studied in other areas of software engineering for a better generalizability of the comparison results.

see all

Copyright information: © Iflaah Salman, 2014. This publication is copyrighted. You may download, display and print it for your own personal use. Commercial use is prohibited.