University of Oulu

Kirill A. Shatilov, Dimitris Chatzopoulos, Lik-Hang Lee, and Pan Hui. 2021. Emerging ExG-based NUI Inputs in Extended Realities: A Bottom-up Survey. ACM Trans. Interact. Intell. Syst. 11, 2, Article 10 (July 2021), 49 pages. DOI:

Emerging ExG-based NUI inputs in extended realities : a bottom-up survey

Saved in:
Author: Shatilov, Kirill A.1; Chatzopoulos, Dimitris1; Lee, Lik-Hang2,3;
Organizations: 1Hong Kong University of Science and Technology, Hong Kong
2KAIST, Republic of Korea
3University of Oulu, Finland
4Hong Kong University of Science and Technology,Hong Kong
5University of Helsinki, Finland
Format: article
Version: published version
Access: open
Online Access: PDF Full Text (PDF, 7.6 MB)
Persistent link:
Language: English
Published: Association for Computing Machinery, 2021
Publish Date: 2021-11-04


Incremental and quantitative improvements of two-way interactions with extended realities (XR) are contributing toward a qualitative leap into a state of XR ecosystems being efficient, user-friendly, and widely adopted. However, there are multiple barriers on the way toward the omnipresence of XR; among them are the following: computational and power limitations of portable hardware, social acceptance of novel interaction protocols, and usability and efficiency of interfaces. In this article, we overview and analyse novel natural user interfaces based on sensing electrical bio-signals that can be leveraged to tackle the challenges of XR input interactions. Electroencephalography-based brain-machine interfaces that enable thought-only hands-free interaction, myoelectric input methods that track body gestures employing electromyography, and gaze-tracking electrooculography input interfaces are the examples of electrical bio-signal sensing technologies united under a collective concept of ExG. ExG signal acquisition modalities provide a way to interact with computing systems using natural intuitive actions enriching interactions with XR. This survey will provide a bottom-up overview starting from (i) underlying biological aspects and signal acquisition techniques, (ii) ExG hardware solutions, (iii) ExG-enabled applications, (iv) discussion on social acceptance of such applications and technologies, as well as (v) research challenges, application directions, and open problems; evidencing the benefits that ExG-based Natural User Interfaces inputs can introduce to the area of XR.

see all

Series: ACM transactions on interactive intelligent systems
ISSN: 2160-6455
ISSN-E: 2160-6463
ISSN-L: 2160-6455
Volume: 11
Issue: 2
Pages: 1 - 49
DOI: 10.1145/3457950
Type of Publication: A1 Journal article – refereed
Field of Science: 113 Computer and information sciences
Funding: This research has been supported in part by 5G-VIIMA and REBOOT Finland IoT Factory projects funded by Business Finland, the 6G Flagship project funded by the Academy of Finland (Decision No. 318927), the 5GEAR project and FIT project from the Academy of Finland, project 16214817 from the Research Grants Council of Hong Kong, as well as Hong Kong Ph.D. Fellowship Scheme.
Academy of Finland Grant Number: 318927
Detailed Information: 318927 (Academy of Finland Funding decision)
Copyright information: © 2021 Copyright held by the owner/author(s). This work is licensed under a Creative Commons Attribution International 4.0 License.