University of Oulu

Niels van Berkel, Jorge Goncalves, Daniel Russo, Simo Hosio, and Mikael B. Skov. 2021. Effect of Information Presentation on Fairness Perceptions of Machine Learning Predictors. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21). Association for Computing Machinery, New York, NY, USA, Article 245, 1–13. DOI:

Effect of information presentation on fairness perceptions of machine learning predictors

Saved in:
Author: van Berkel, Niels1; Goncalves, Jorge2; Russo, Daniel1;
Organizations: 1Aalborg University, Aalborg, Denmark
2University of Melbourne, Melbourne, Australia
3University of Oulu, Oulu, Finland
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 1.6 MB)
Persistent link:
Language: English
Published: Association for Computing Machinery, 2021
Publish Date: 2021-05-21


The uptake of artificial intelligence-based applications raises concerns about the fairness and transparency of AI behaviour. Consequently, the Computer Science community calls for the involvement of the general public in the design and evaluation of AI systems. Assessing the fairness of individual predictors is an essential step in the development of equitable algorithms. In this study, we evaluate the effect of two common visualisation techniques (text-based and scatterplot) and the display of the outcome information (i.e., ground-truth) on the perceived fairness of predictors. Our results from an online crowdsourcing study (N = 80) show that the chosen visualisation technique significantly alters people’s fairness perception and that the presented scenario, as well as the participant’s gender and past education, influence perceived fairness. Based on these results we draw recommendations for future work that seeks to involve non-experts in AI fairness evaluations.

see all

ISBN Print: 978-1-4503-8096-6
Article number: 245
DOI: 10.1145/3411764.3445365
Host publication: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
Conference: ACM SIGCHI Annual Conference on Human Factors in Computing Systems
Type of Publication: A4 Article in conference proceedings
Field of Science: 113 Computer and information sciences
Funding: We would like to express our gratitude to our study participants and the reviewers of this manuscript. This research is partially funded by the GenZ strategic profling theme at the University of Oulu, supported by the Academy of Finland (project number 318930).
Academy of Finland Grant Number: 318930
Detailed Information: 318930 (Academy of Finland Funding decision)
Copyright information: © 2021 Copyright held by the owner/author(s). Publication rights licensed to ACM. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems,