University of Oulu

Hettiachchi D., van Berkel N., Hosio S., Kostakos V., Goncalves J. (2019) Effect of Cognitive Abilities on Crowdsourcing Task Performance. In: Lamas D., Loizides F., Nacke L., Petrie H., Winckler M., Zaphiris P. (eds) Human-Computer Interaction – INTERACT 2019. INTERACT 2019. Lecture Notes in Computer Science, vol 11746. Springer, Cham. https://doi.org/10.1007/978-3-030-29381-9_28

Effect of cognitive abilities on crowdsourcing task performance

Saved in:
Author: Hettiachchi, Danula1; van Berkel, Niels1; Hosio, Simo2;
Organizations: 1School of Computing and Information Systems, The University of Melbourne, Australia
2Center for Ubiquitous Computing, University of Oulu, Finland
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 0.8 MB)
Persistent link: http://urn.fi/urn:nbn:fi-fe2020110689607
Language: English
Published: Springer Nature, 2019
Publish Date: 2020-11-06
Description:

Abstract

Matching crowd workers to suitable tasks is highly desirable as it can enhance task performance, reduce the cost for requesters, and increase worker satisfaction. In this paper, we propose a method that considers workers’ cognitive ability to predict their suitability for a wide range of crowdsourcing tasks. We measure cognitive ability via fast-paced online cognitive tests with a combined average duration of 6.2 min. We then demonstrate that our proposed method can effectively assign or recommend workers to five different popular crowd tasks: Classification, Counting, Proofreading, Sentiment Analysis, and Transcription. Using our approach we demonstrate a significant improvement in the expected overall task accuracy. While previous methods require access to worker history or demographics, our work offers a quick and accurate way to determine which workers are more suitable for which tasks.

see all

Series: Lecture notes in computer science
ISSN: 0302-9743
ISSN-E: 1611-3349
ISSN-L: 0302-9743
ISBN Print: 978-3-030-29380-2
Pages: 442 - 464
DOI: 10.1007/978-3-030-29381-9_28
OADOI: https://oadoi.org/10.1007/978-3-030-29381-9_28
Host publication: Human-Computer Interaction – INTERACT 2019 17th IFIP TC 13 International Conference, Paphos, Cyprus, September 2–6, 2019, Proceedings, Part I
Host publication editor: Lamas, David
Loizides, Fernando
Nacke, Lennart
Petrie, Helen
Winckler, Marco
Zaphiris, Panayiotis
Conference: IFIP International Conference on Human-Computer Interaction
Type of Publication: A4 Article in conference proceedings
Field of Science: 113 Computer and information sciences
Subjects:
Copyright information: © IFIP International Federation for Information Processing 2019. This is a post-peer-review, pre-copyedit version of an article published in Human-Computer Interaction – INTERACT 2019 17th IFIP TC 13 International Conference, Paphos, Cyprus, September 2–6, 2019, Proceedings, Part I. The final authenticated version is available online at: https://doi.org/10.1007/978-3-030-29381-9_28.