JultikaUniversity of Oulu repository

Abstract

Chatbots have become an omnipresent software that many services are using nowadays to provide easy and continuous support for users. Regardless of the domain in question, people are using chatbots to get quick access to information in a human-like manner. Still, chatbots are limited in terms of interactivity, providing facts, or solving elemental problems. Moreover, the lack of empathy that chatbots have is a drawback that limits them from providing the best outcome possible for the user.

With that in mind, this thesis aims to find out how an emotionally aware chatbot would influence the interaction and engagement level of participants, starting from the hypothesis that “The awareness that the chatbot shows during the conversation impacts the engagement of participants”. The research method used was an experimental study approach because it helps with finding how the cause of the awareness of chatbots can affect the engagement level of participants. For that, a web application was developed that consisted of a chatbot driven by OpenAI. Before the participants started to interact with the chatbot, they were provided with information and instructions on how to adjust their cameras so their facial expressions could be analyzed properly in order to get the intended experience. A total number of 180 participants were recruited using the Prolific crowd-sourcing platform, from which 178 responses were used in analyzing the results.

The participants were split into three study conditions, namely BASELINE, EMOJIONLY, and EMOJI-AND-CHAT which differed in the emotional awareness levels that the chatbot had. BASELINE study group interacted with a simple chatbot that was not aware of participants’ emotions at all. The EMOJI-ONLY study group discussed with a chatbot that during the interaction showed participants their emotions in real time with the use of emoji pictograms. In the last study group, EMOJI-ANDCHAT, besides showing the participants’ expressions through emojis, the chatbot also replied to the mood changes of the participants with messages that clearly stated that the chatbot noticed their facial expression changes. Each participant, regardless of the study group, had a conversation with the chatbot that lasted for a few minutes and started with the topic of their own chronic pain experiences. The chronic pain topic was used in order to trigger differences in facial expressions naturally. During a conversation of only a few minutes, the topic discussed needs to be of interest to the participant so that differences in facial expressions could occur. With that in mind, participants were recruited using Prolific’s option of selecting participants that deal with chronic pain. During the conversations participants’ facial expressions were analyzed and collected. Moreover, at the end of the interaction, the participants answered a questionnaire composed of a mix of 23 quantitative and 3 qualitative questions.

The data collected showed that the emotional awareness that a chatbot is showing during a discussion impacts the level of engagement of participants. However, the results were not able to particularly point out if participants’ level of engagement is affected positively, and thus feeling more engaged, or is affected negatively, feeling less engaged than when interacting with a non-emotional aware chatbot. Participants showed both significant interests in the emotionally aware chatbots, as well as concerns, and identified possible issues and limitations.

The chatbot used throughout this research was effective and succeeded to show the potential of such applications. Nevertheless, improving the way the chatbot reacts to changes in facial expressions needs further testing and development, as well as improving its privacy and security side so people would trust it more.