University of Oulu

Martos, O., Hoque, M. Z., Keskinarkaus, A., Kemi, N., Näpänkangas, J., Eskuri, M., Pohjanen, V.-M., Kauppila, J. H., & Seppänen, T. (2023). Optimized detection and segmentation of nuclei in gastric cancer images using stain normalization and blurred artifact removal. In Pathology - Research and Practice (Vol. 248, p. 154694). Elsevier BV.

Optimized detection and segmentation of nuclei in gastric cancer images using stain normalization and blurred artifact removal

Saved in:
Author: Martos, Oleg1; Hoque, Md. Ziaul1; Keskinarkaus, Anja1;
Organizations: 1Center for Machine Vision and Signal Analysis, Faculty of Information Technology and Electrical Engineering, University of Oulu, Finland
2Department of Pathology, Oulu University Hospital, Finland, and University of Oulu, Finland
3Department of Surgery, Oulu University Hospital, Finland, and University of Oulu, Finland
Format: article
Version: published version
Access: open
Online Access: PDF Full Text (PDF, 14 MB)
Persistent link:
Language: English
Published: Elsevier, 2023
Publish Date: 2023-10-12


Histological analysis with microscopy is the gold standard to diagnose and stage cancer, where slides or whole slide images are analyzed for cell morphological and spatial features by pathologists. The nuclei of cancerous cells are characterized by nonuniform chromatin distribution, irregular shapes, and varying size. As nucleus area and shape alone carry prognostic value, detection and segmentation of nuclei are among the most important steps in disease grading. However, evaluation of nuclei is a laborious, time-consuming, and subjective process with large variation among pathologists. Recent advances in digital pathology have allowed significant applications in nuclei detection, segmentation, and classification, but automated image analysis is greatly affected by staining factors, scanner variability, and imaging artifacts, requiring robust image preprocessing, normalization, and segmentation methods for clinically satisfactory results. In this paper, we aimed to evaluate and compare the digital image analysis techniques used in clinical pathology and research in the setting of gastric cancer. A literature review was conducted to evaluate potential methods of improving nuclei detection. Digitized images of 35 patients from a retrospective cohort of gastric adenocarcinoma at Oulu University Hospital in 1987–2016 were annotated for nuclei (n = 9085) by expert pathologists and 14 images of different cancer types from public TCGA dataset with annotated nuclei (n = 7000) were used as a comparison to evaluate applicability in other cancer types. The detection and segmentation accuracy with the selected color normalization and stain separation techniques were compared between the methods. The extracted information can be supplemented by patient’s medical data and fed to the existing statistical clinical tools or subjected to subsequent AI-assisted classification and prediction models. The performance of each method is evaluated by several metrics against the annotations done by expert pathologists. The F1-measure of 0.854 ± 0.068 is achieved with color normalization for the gastric cancer dataset, and 0.907 ± 0.044 with color deconvolution for the public dataset, showing comparable results to the earlier state-of-the-art works. The developed techniques serve as a basis for further research on application and interpretability of AI-assisted tools for gastric cancer diagnosis.

see all

Series: Pathology. Research and practice
ISSN: 0344-0338
ISSN-E: 1618-0631
ISSN-L: 0344-0338
Volume: 248
Article number: 154694
DOI: 10.1016/j.prp.2023.154694
Type of Publication: A1 Journal article – refereed
Field of Science: 113 Computer and information sciences
217 Medical engineering
3122 Cancers
Funding: This research has been financially supported by Academy of Finland 6 Genesis Flagship (Grant 318927) and Academy of Finland Identifying trajectories of healthy aging via integration of birth cohorts and biobank data (Grant 309112).
Academy of Finland Grant Number: 318927
Detailed Information: 318927 (Academy of Finland Funding decision)
309112 (Academy of Finland Funding decision)
Copyright information: © 2023 The Author(s). Published by Elsevier GmbH. This is an open access article under the CC BY license (