University of Oulu

M. Suomalainen, A. Q. Nilles and S. M. LaValle, "Virtual Reality for Robots," 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA, 2020, pp. 11458-11465, doi: 10.1109/IROS45743.2020.9341344

Virtual reality for robots

Saved in:
Author: Suomalainen, Markku1; Nilles, Alexandra Q.2; LaValle, Steven M.1
Organizations: 1Center of Ubiquitous Computing, Faculty of Information Technology and Electrical Engineering, University of Oulu, Finland
2Department of Computer Science, University of Illinois at Urbana-Champaign, Illinois, USA
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 1 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2020
Publish Date: 2021-02-23


This paper applies the principles of Virtual Reality (VR) to robots, rather than living organisms. A simulator, of either physical states or information states, renders outputs to custom displays that fool the robot’s sensors. This enables a robot to experience a combination of real and virtual sensor inputs, combining the efficiency of simulation and the benefits of real world sensor inputs. Thus, the robot can be taken through targeted experiences that are more realistic than pure simulation, yet more feasible and controllable than pure real-world experiences. We define two distinctive methods for applying VR to robots, namely black box and white box; based on these methods we identify potential applications, such as testing and verification procedures that are better than simulation, the study of spoofing attacks and anti-spoofing techniques, and sample generation for machine learning. A general mathematical framework is presented, along with a simple experiment, detailed examples, and discussion of the implications.

see all

Series: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems
ISSN: 2153-0858
ISSN-E: 2153-0866
ISSN-L: 2153-0858
ISBN: 978-1-7281-6212-6
ISBN Print: 978-1-7281-6213-3
Pages: 11458 - 11465
DOI: 10.1109/IROS45743.2020.9341344
Host publication: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Conference: IEEE/RSJ International Conference on Intelligent Robots and Systems
Type of Publication: A4 Article in conference proceedings
Field of Science: 213 Electronic, automation and communications engineering, electronics
113 Computer and information sciences
Funding: This work was supported by Business Finland project HUMORcc 6926/31/2018, Academy of Finland project PERCEPT, 322637 and US National Science Foundation grants 035345, 1328018.
Academy of Finland Grant Number: 322637
Detailed Information: 322637 (Academy of Finland Funding decision)
Copyright information: © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.