Effect of experience sampling schedules on response rate and recall accuracy of objective self-reports
van Berkel, Niels; Goncalves, Jorge; Lovén, Lauri; Ferreira, Denzil; Hosio, Simo; Kostakos, Vassilis (2018-12-03)
Niels van Berkel, Jorge Goncalves, Lauri Lovén, Denzil Ferreira, Simo Hosio, Vassilis Kostakos, Effect of experience sampling schedules on response rate and recall accuracy of objective self-reports, International Journal of Human-Computer Studies, Volume 125, 2019, Pages 118-128, ISSN 1071-5819, https://doi.org/10.1016/j.ijhcs.2018.12.002
© 2018. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/
https://creativecommons.org/licenses/by-nc-nd/4.0/
https://urn.fi/URN:NBN:fi-fe2018121951344
Tiivistelmä
Abstract
The Experience Sampling Method is widely used to collect human labelled data in the wild. Using this methodology, study participants repeatedly answer a set of questions, constructing a rich overview of the studied phenomena. One of the methodological decisions faced by researchers is deciding on the question scheduling. The literature defines three distinct schedule types: randomised, interval-based, or event-based (in our case, smartphone unlock). However, little evidence exists regarding the side-effects of these schedules on response rate and recall accuracy, and how they may bias study findings. We evaluate the effect of these three contingency configurations in a 3-week within-subjects study (N=20). Participants answered various objective questions regarding their phone usage, while we simultaneously establish a ground-truth through smartphone instrumentation. We find that scheduling questions on phone unlock yields a higher response rate and accuracy. Our study provides empirical evidence for the effects of notification scheduling on participant responses, and informs researchers who conduct experience sampling studies on smartphones.
Kokoelmat
- Avoin saatavuus [31941]