van Berkel, N., Goncalves, J., Lovén, L., Ferreira, D., Hosio, S., Kostakos, V. (2018) Effect of experience sampling schedules on response rate and recall accuracy of objective self-reports. International Journal of Human-Computer Studies. doi:10.1016/j.ijhcs.2018.12.002
Effect of experience sampling schedules on response rate and recall accuracy of objective self-reports
|Author:||van Berkel, Niels1; Goncalves, Jorge1; Lovén, Lauri2;|
1The University of Melbourne
2University of Oulu
|Persistent link:|| http://urn.fi/urn:nbn:fi-fe2018121951344
|Publish Date:|| 2020-12-03
The Experience Sampling Method is widely used to collect human labelled data in the wild. Using this methodology, study participants repeatedly answer a set of questions, constructing a rich overview of the studied phenomena. One of the methodological decisions faced by researchers is deciding on the question scheduling. The literature defines three distinct schedule types: randomised, interval-based, or event-based (in our case, smartphone unlock). However, little evidence exists regarding the side-effects of these schedules on response rate and recall accuracy, and how they may bias study findings. We evaluate the effect of these three contingency configurations in a 3-week within-subjects study (N=20). Participants answered various objective questions regarding their phone usage, while we simultaneously establish a ground-truth through smartphone instrumentation. We find that scheduling questions on phone unlock yields a higher response rate and accuracy. Our study provides empirical evidence for the effects of notification scheduling on participant responses, and informs researchers who conduct experience sampling studies on smartphones.
International journal of human-computer studies
|Type of Publication:||
A1 Journal article – refereed
|Field of Science:||
113 Computer and information sciences
This work is partially funded by the Academy of Finland (Grants 276786-AWARE, 286386-CPDSS, 285459-iSCIENCE, 304925-CARE), the European Commission (Grant 6AIKA-A71143-AKAI), and Marie Skłodowska-Curie Actions (645706-GRAGE).
|EU Grant Number:||
(645706) GRAGE - Grey and green in Europe: elderly living in urban areas
|Academy of Finland Grant Number:||
276786 (Academy of Finland Funding decision)
286386 (Academy of Finland Funding decision)
285459 (Academy of Finland Funding decision)
304925 (Academy of Finland Funding decision)
© 2018 Elsevier Ltd. All rights reserved. Published in this repository with the kind permission of the publisher.