RF-inpainter : multimodal image inpainting based on vision and radio signals
Chen, Cheng; Nishio, Takayuki; Bennis, Mehdi; Park, Jihong (2022-10-14)
C. Chen, T. Nishio, M. Bennis and J. Park, "RF-Inpainter: Multimodal Image Inpainting Based on Vision and Radio Signals," in IEEE Access, vol. 10, pp. 110689-110700, 2022, doi: 10.1109/ACCESS.2022.3214972
© The Author(s) 2022. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
https://creativecommons.org/licenses/by/4.0/
https://urn.fi/URN:NBN:fi-fe202301173368
Tiivistelmä
Abstract
This study demonstrates the feasibility of image inpainting using both visual information and radio frequency (RF) signals. Recent developments in imaging and vision-based technologies using RF signals have revealed the potential of leveraging multimodal information to enhance image inpainting performance. In this context, we propose RF-Inpainter—a novel inpainting method that integrates visual and wireless information by fusing defective RGB images with received signal strength indicator (RSSI) using a deep auto-encoder model. The inpainting performance of RF-Inpainter is evaluated using experimentally obtained images and RSSI datasets in an indoor environment. Image-only inpainting and RSSI-only inpainting models are used as baselines to illustrate the superiority of RF-Inpainter over inpainting methods based on a single modality. The results establish that RF-Inpainter generates satisfactory inpainted images in most experimental scenarios, achieving a maximum improvement of 36.4% and 14.6% in terms of mean peak signal-to-noise ratio (PSNR) and mean structural similarity index (SSIM), respectively.
Kokoelmat
- Avoin saatavuus [31657]