University of Oulu

Jorge Goncalves, Simo Hosio, and Vassilis Kostakos. 2017. Eliciting structured knowledge from situated crowd markets. ACM Trans. Internet Technol. 17, 2, Article 14 (March 2017), 21 pages. DOI:

Eliciting structured knowledge from situated crowd markets

Saved in:
Author: Goncalves, Jorge1,2; Hosio, Simo1; Kostakos, Vassilis1,2
Organizations: 1Center for Ubiquitous Computing, University of Oulu
2Department of Computing and Information Systems, The University of Melbourne
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 0.8 MB)
Persistent link:
Language: English
Published: Association for Computing Machinery, 2017
Publish Date: 2019-04-05


We present a crowdsourcing methodology to elicit highly structured knowledge for arbitrary questions. The method elicits potential answers (“options”), criteria against which those options should be evaluated, and a ranking of the top “options.” Our study shows that situated crowdsourcing markets can reliably elicit/moderate knowledge to generate a ranking of options based on different criteria that correlate with established online platforms. Our evaluation also shows that local crowds can generate knowledge that is missing from online platforms and on how a local crowd perceives a certain issue. Finally, we discuss the benefits and challenges of eliciting structured knowledge from local crowds.

see all

Series: ACM transactions on internet technology
ISSN: 1533-5399
ISSN-E: 1557-6051
ISSN-L: 1533-5399
Volume: 17
Issue: 2
Article number: 14
DOI: 10.1145/3007900
Type of Publication: A1 Journal article – refereed
Field of Science: 113 Computer and information sciences
Funding: This work is partially funded by the Academy of Finland (Grants 276786-AWARE, 285062-iCYCLE, 286386-CPDSS, and 285459-iSCIENCE) and the European Commission (Grants PCIG11-GA-2012-322138, 645706-GRAGE, and 6AIKA-A71143-AKAI).
EU Grant Number: (645706) GRAGE - Grey and green in Europe: elderly living in urban areas
(322138) URBANMOB - Modelling Urban Mobility in City-Scale Ubiquitous Systems
Academy of Finland Grant Number: 276786
Detailed Information: 276786 (Academy of Finland Funding decision)
285062 (Academy of Finland Funding decision)
286386 (Academy of Finland Funding decision)
285459 (Academy of Finland Funding decision)
Copyright information: © ACM 2017. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ACM transactions on internet technology,