Djamila-Romaissa, B., Oussalah, M., Seppänen, T., Attention-based CNN-GRU model for automatic medical images captioning : ImageCLEF 2021, 2021 Working Notes of CLEF - Conference and Labs of the Evaluation Forum, CLEF-WN 2021, 21-24 September, Bucharest, Romania, 1613-0073, 1160-1173
Attention-based CNN-GRU model for automatic medical images captioning : ImageCLEF 2021
|Author:||Djamila-Romaissa, Beddiar1; Oussalah, Mourad1,2; Seppänen, Tapio1|
1Center for machine vision and signal analysis, University of Oulu, Finland
2MIPT, Faculty of Medicine, University of Oulu, Oulu, Finland
|Online Access:||PDF Full Text (PDF, 1.4 MB)|
|Persistent link:|| http://urn.fi/urn:nbn:fi-fe2021102151849
RWTH Aachen University,
|Publish Date:|| 2021-10-21
The action of understanding and interpretation of medical images is a very important task in the medical diagnosis generation. However, manual description of medical content is a major bottleneck in clinical diagnosis. Many research studies were devoted to develop automated alternatives to this process, which would have enormous impact in terms of efficiency, cost and accuracy in the clinical workflows. Different approaches and techniques have been presented in the literature ranging from traditional machine learning methods to deep learning based models. Inspired by the outperforming results of the later techniques, we present in the current paper, our team participation (RomiBed) to the ImageCLEF medical caption prediction task. We addressed the challenge of medical image captioning by combining a CNN encoder model with an attention-based GRU language generator model whereas a multi-label CNN classifier is used for the concept detection task. Using the provided data in the training, validation and test subsets, we obtain an average F_measure of 14.3% and a BLEU score of 0.243 on the ImageCLEF concept detection and the caption prediction challenges, respectively.
CEUR workshop proceedings
|Pages:||1160 - 1173|
2021 Working Notes of CLEF - Conference and Labs of the Evaluation Forum, CLEF-WN 2021, 21-24 September, Bucharest, Romania
|Host publication editor:||
Conference and Labs of the Evaluation Forum
|Type of Publication:||
A4 Article in conference proceedings
|Field of Science:||
113 Computer and information sciences
This work is supported by the Academy of Finland Profi5 DigiHealth project (#326291), which is gratefully acknowledged.
© 2021 for the individual papers by the papers' authors. Copyright © 2021 for the volume as a collection by its editors. This volume and its papers are published under the Creative Commons License Attribution 4.0 International (CC BY 4.0).