University of Oulu

Martynov, P., Mitropolskii, N., Kukkola, K., Gretsch, M., Koivisto, V., Lindgren, I., Saunavaara, J., Reponen, J., Mäkynen, A. (2017) Testing of the assisting software for radiologists analysing head CT images: lessons learned. BMC Medical Imaging, 17 (1). doi:10.1186/s12880-017-0229-1

Testing of the assisting software for radiologists analysing head CT images : lessons learned

Saved in:
Author: Martynov, Petr1; Mitropolskii, Nikolai1; Kukkola, Katri1;
Organizations: 1Optoelectronics and Measurement Techniques Unit, University of Oulu
2Department of Radiology, Oulu University Hospital
3Department of Radiology, Lapland Hospital District
4Medical Imaging Centre of Southwest Finland, Turku University Hospital
5Finntelemedicum, Research Unit of Medical Imaging, Physics and Technology, University of Oulu
6Department of Radiology, Hospital of Raahe
Format: article
Version: published version
Access: open
Online Access: PDF Full Text (PDF, 1.9 MB)
Persistent link:
Language: English
Published: Springer Nature, 2017
Publish Date: 2017-12-15


Background: Assessing a plan for user testing and evaluation of the assisting software developed for radiologists.

Methods: Test plan was assessed in experimental testing, where users performed reporting on head computed tomography studies with the aid of the software developed. The user testing included usability tests, questionnaires, and interviews. In addition, search relevance was assessed on the basis of user opinions.

Results: The testing demonstrated weaknesses in the initial plan and enabled improvements. Results showed that the software has acceptable usability level but some minor fixes are needed before larger-scale pilot testing. The research also proved that it is possible even for radiologists with under a year’s experience to perform reporting of non-obvious cases when assisted by the software developed. Due to the small number of test users, it was impossible to assess effects on diagnosis quality.

Conclusions: The results of the tests performed showed that the test plan designed is useful, and answers to the key research questions should be forthcoming after testing with more radiologists. The preliminary testing revealed opportunities to improve test plan and flow, thereby illustrating that arranging preliminary test sessions prior to any complex scenarios is beneficial.

see all

Series: BMC medical imaging
ISSN: 1471-2342
ISSN-E: 1471-2342
ISSN-L: 1471-2342
Volume: 17
Issue: 1
Article number: 59
DOI: 10.1186/s12880-017-0229-1
Type of Publication: A1 Journal article – refereed
Field of Science: 113 Computer and information sciences
217 Medical engineering
3126 Surgery, anesthesiology, intensive care, radiology
Funding: The entire CARDS project was funded by Tekes – the Finnish Funding Agency for Innovation, decision number 2490/31/2014. The funding body have not influenced in any way the design of the study, collection, analysis, and interpretation of data and the manuscript writing.
Copyright information: © The Author(s) 2017. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.