University of Oulu

C. Chen, T. Nishio, M. Bennis and J. Park, "RF-Inpainter: Multimodal Image Inpainting Based on Vision and Radio Signals," in IEEE Access, vol. 10, pp. 110689-110700, 2022, doi: 10.1109/ACCESS.2022.3214972

RF-inpainter : multimodal image inpainting based on vision and radio signals

Saved in:
Author: Chen, Cheng1; Nishio, Takayuki1; Bennis, Mehdi2;
Organizations: 1School of Engineering, Tokyo Institute of Technology, Ookayama, Meguro-ku, Tokyo 152-8550, Japan
2Centre of Wireless Communications, University of Oulu, 90014 Oulu, Finland
3School of Info Technology, Deakin University, Geelong, VIC 3220, Australia
Format: article
Version: published version
Access: open
Online Access: PDF Full Text (PDF, 2.7 MB)
Persistent link: http://urn.fi/urn:nbn:fi-fe202301173368
Language: English
Published: Institute of Electrical and Electronics Engineers, 2022
Publish Date: 2023-01-17
Description:

Abstract

This study demonstrates the feasibility of image inpainting using both visual information and radio frequency (RF) signals. Recent developments in imaging and vision-based technologies using RF signals have revealed the potential of leveraging multimodal information to enhance image inpainting performance. In this context, we propose RF-Inpainter—a novel inpainting method that integrates visual and wireless information by fusing defective RGB images with received signal strength indicator (RSSI) using a deep auto-encoder model. The inpainting performance of RF-Inpainter is evaluated using experimentally obtained images and RSSI datasets in an indoor environment. Image-only inpainting and RSSI-only inpainting models are used as baselines to illustrate the superiority of RF-Inpainter over inpainting methods based on a single modality. The results establish that RF-Inpainter generates satisfactory inpainted images in most experimental scenarios, achieving a maximum improvement of 36.4% and 14.6% in terms of mean peak signal-to-noise ratio (PSNR) and mean structural similarity index (SSIM), respectively.

see all

Series: IEEE access
ISSN: 2169-3536
ISSN-E: 2169-3536
ISSN-L: 2169-3536
Volume: 10
Pages: 110689 - 110700
DOI: 10.1109/ACCESS.2022.3214972
OADOI: https://oadoi.org/10.1109/ACCESS.2022.3214972
Type of Publication: A1 Journal article – refereed
Field of Science: 213 Electronic, automation and communications engineering, electronics
Subjects:
Funding: This work was supported in part by JSPS KAKENHI under Grant JP22H03575.
Copyright information: © The Author(s) 2022. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
  https://creativecommons.org/licenses/by/4.0/