University of Oulu

Djamila-Romaissa, B., Oussalah, M., Seppänen, T., Attention-based CNN-GRU model for automatic medical images captioning : ImageCLEF 2021, 2021 Working Notes of CLEF - Conference and Labs of the Evaluation Forum, CLEF-WN 2021, 21-24 September, Bucharest, Romania, 1613-0073, 1160-1173

Attention-based CNN-GRU model for automatic medical images captioning : ImageCLEF 2021

Saved in:
Author: Djamila-Romaissa, Beddiar1; Oussalah, Mourad1,2; Seppänen, Tapio1
Organizations: 1Center for machine vision and signal analysis, University of Oulu, Finland
2MIPT, Faculty of Medicine, University of Oulu, Oulu, Finland
Format: article
Version: published version
Access: open
Online Access: PDF Full Text (PDF, 1.4 MB)
Persistent link: http://urn.fi/urn:nbn:fi-fe2021102151849
Language: English
Published: RWTH Aachen University, 2021
Publish Date: 2021-10-21
Description:

Abstract

The action of understanding and interpretation of medical images is a very important task in the medical diagnosis generation. However, manual description of medical content is a major bottleneck in clinical diagnosis. Many research studies were devoted to develop automated alternatives to this process, which would have enormous impact in terms of efficiency, cost and accuracy in the clinical workflows. Different approaches and techniques have been presented in the literature ranging from traditional machine learning methods to deep learning based models. Inspired by the outperforming results of the later techniques, we present in the current paper, our team participation (RomiBed) to the ImageCLEF medical caption prediction task. We addressed the challenge of medical image captioning by combining a CNN encoder model with an attention-based GRU language generator model whereas a multi-label CNN classifier is used for the concept detection task. Using the provided data in the training, validation and test subsets, we obtain an average F_measure of 14.3% and a BLEU score of 0.243 on the ImageCLEF concept detection and the caption prediction challenges, respectively.

see all

Series: CEUR workshop proceedings
ISSN: 1613-0073
ISSN-E: 1613-0073
ISSN-L: 1613-0073
Pages: 1160 - 1173
Host publication: 2021 Working Notes of CLEF - Conference and Labs of the Evaluation Forum, CLEF-WN 2021, 21-24 September, Bucharest, Romania
Host publication editor: Faggioli, Guglielmo
Ferro, Nicola
Joly, Alexis
Maistro, Maria
Piroi, Florina
Conference: Conference and Labs of the Evaluation Forum
Type of Publication: A4 Article in conference proceedings
Field of Science: 113 Computer and information sciences
Subjects:
Funding: This work is supported by the Academy of Finland Profi5 DigiHealth project (#326291), which is gratefully acknowledged.
Academy of Finland Grant Number: 326291
Detailed Information: 326291 (Academy of Finland Funding decision)
Copyright information: © 2021 for the individual papers by the papers' authors. Copyright © 2021 for the volume as a collection by its editors. This volume and its papers are published under the Creative Commons License Attribution 4.0 International (CC BY 4.0).
  https://creativecommons.org/licenses/by/4.0/