University of Oulu

J. F. Abascal et al., "Material Decomposition in Spectral CT using deep learning: A Sim2Real transfer approach," in IEEE Access, doi: 10.1109/ACCESS.2021.3056150

Material decomposition in spectral CT using deep learning : a Sim2Real transfer approach

Saved in:
Author: Abascal, Juan Fpj1; Ducros, Nicolas1; Pronina, Valeriya2,1;
Organizations: 1Univ Lyon, INSA-Lyon, CNRS, Inserm, CREATIS UMR 5220, U1206, Lyon, France
2Skolkovo Institute of Science and Technology, Moscow, Russia
3Research Unit of Mathematical Sciences, University of Oulu, Oulu, Finland
4Department of Computer Science, University College London, London, United Kingdom
Format: article
Version: published version
Access: open
Online Access: PDF Full Text (PDF, 18.3 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2021
Publish Date: 2021-02-09


The state-of-the art for solving the nonlinear material decomposition problem in spectral computed tomography is based on variational methods, but these are computationally slow and critically depend on the particular choice of the regularization functional. Convolutional neural networks have been proposed for addressing these issues. However, learning algorithms require large amounts of experimental data sets. We propose a deep learning strategy for solving the material decomposition problem based on a U-Net architecture and a Sim2Real transfer learning approach where the knowledge that we learn from synthetic data is transferred to a real-world scenario. In order for this approach to work, synthetic data must be realistic and representative of the experimental data. For this purpose, numerical phantoms are generated from human CT volumes of the KiTS19 Challenge dataset, segmented into specific materials (soft tissue and bone). These volumes are projected into sinogram space in order to simulate photon counting data, taking into account the energy response of the scanner. We compared projection- and image-based decomposition approaches where the network is trained to decompose the materials either in the projection or in the image domain. The proposed Sim2Real transfer strategies are compared to a regularized Gauss-Newton (RGN) method on synthetic data, experimental phantom data and human thorax data.

see all

Series: IEEE access
ISSN: 2169-3536
ISSN-E: 2169-3536
ISSN-L: 2169-3536
Volume: Early Access
DOI: 10.1109/ACCESS.2021.3056150
Type of Publication: A1 Journal article – refereed
Field of Science: 113 Computer and information sciences
217 Medical engineering
Funding: This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 701915. It was also performed within the framework of the LabEx PRIMES (ANR-11-LABX-0063) of University de Lyon and under the support of the ANR project SALTO (ANR-17-CE19-0011-01). The project has also received funding from the European Union’s Horizon 2020 research and innovation programme under the grant agreement N 668142. This work was partly funded by France Life Imaging (grant ANR-11-INBS-0006) from the French Investissements d’Avenir. This work was partially supported by the Academy of Finland Project 312123 (Finnish Centre of Excellence in Inverse Modelling and Imaging, 2018–2025). SA acknowledges support of EPSRC grants EP/N022750/1 and EP/T000864/1.
Academy of Finland Grant Number: 312123
Detailed Information: 312123 (Academy of Finland Funding decision)
Copyright information: © 2021 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see