Emergency department triage: Artificial Intelligence’s gateway to radiology

The earliest adoption of artificial intelligence (AI) within clinical workflows has emerged within the emergency setting where it can manage priority for interpretation of imaging studies. In this triage role, AI does not commit to a diagnosis; rather it offers a binary decision as to whether the image contains a specific finding. The goal is to expedite the interpretation of the most critical cases, ultimately leading to improved patient outcomes.

Stroke workflow, primed for optimisation
One clinical domain particularly suited to workflow optimisation, due to its time critical nature, is acute stroke. Advances in treatment have resulted in continually shifting guidelines, adding complexity to the time pressured decisions. With several key imaging features involved in stroke triage, it lends itself to the current focus on narrow AI solutions.

Intracranial Hemorrhage (ICH) detection
ICH is a medical emergency and timely diagnosis is critical as nearly half of resulting mortalities occur within the first 24 hours. The speed of interpretation is dependent on the priority assigned to the scan request, which is a particular risk when symptoms can be vague. Automated ICH detection, as implemented by Canon Medical Systems’ Stroke CT Package, can address this problem by automatically detecting ICH and pushing the results to the neurointerventionalist. A case example is shown in Figure 1.

Performance of this algorithm, assessed in a validation cohort of 200 ICH positive and 102 non-ICH patients, yielded the following results (Table 1): a sensitivity of 0.93, specificity of 0.93, Positive Predictive Value (PPV) of 0.85 and Negative Predictive Value (NPV) of 0.98. Of note, where the algorithm performance is challenged is in cases of small volume hemorrhages. The author notes that ensemble methods using multimodal data may be used to address this limitation in the future.

	All (n = 258)	Small ICH (n = 93)	Medium ICH (n = 117)	Large ICH (n = 48)
ICH volume (mL)	17.2 ± 2.7	1.7 ± 0.3	13.2 ± 1.2	57.3 ± 6.1
Accuracy	0.94 ± 0.01	0.94 ± 0.02	0.93 ± 0.02	0.95 ± 0.02
Sensitivity	0.93 ± 0.03	0.89 ± 0.05	0.94 ± 0.04	0.99 ± 0.01
Specificity	0.93 ± 0.01	0.94 ± 0.02	0.92 ± 0.02	0.92 ± 0.04
Positive predictive value	0.85 ± 0.02	0.81 ± 0.05	0.86 ± 0.03	0.91 ± 0.04
Negative predictive value	0.98 ± 0.01	0.98 ± 0.01	0.98 ± 0.01	0.99 ± 0.01
F1 score	0.86 ± 0.03	0.81 ± 0.06	0.87 ± 0.04	0.94 ± 0.02
Matthews correlation coefficient	0.87 ± 0.02	0.83 ± 0.04	0.87 ± 0.03	0.90 ± 0.04
Proper triage as ICH positive, % (n)	95 (245)	92.5 (86)	94.9 (111)	100.0 (48)

Table 1. 95% Confidence Intervals for ICH volume, accuracy, sensitivity, specificity, positive predictive value, negative predictive value, F1 score, and Matthews correlation metrics corresponding to the ICH detection algorithm for all, small (≤5 mL), medium (>5 and <30 mL), and large (≥30 mL) ICHs. The percentage of times the algorithm correctly detects an ICH is also indicated.

Reference
Rava et al. | Assessment of an Artificial Intelligence Algorithm for Detection of Intracranial Hemorrhage | World Neurosurgery (2021)
https://pubmed.ncbi.nlm.nih.gov/33684578/

Large Vessel Occlusion (LVO) detection
With the introduction of endovascular clot retrieval to routine clinical workflows, the identification of those patients who would benefit from the treatment quickly became top priority in stroke workflows. Implementing the automated detection of large vessel occlusions (LVO) in CTA, as provided by Canon Medical Systems’ Stroke CT Package, addresses this triage need.

The performance of this LVO detection solution was assessed in a cohort of 202 acute ischemic patients, 100 of whom had an occlusion within the Internal Carotid Artery (ICA), M1 or M2 regions of the Middle Cerebral Artery (MCA) and 102 patients with no occlusion. Analysis including all patients produced the following metrics (Table 1): a sensitivity of 0.73, specificity of 0.98, PPV of 0.99 and NPV of 0.64. As with ICH detection, it seems size matters. As the occlusions become more distal, within the MCA M2 region, they decrease in size and a drop off in performance is seen, with sensitivity falling to 0.5. With any such automated detection task there is a trade off in sensitivity versus specificity, however in this clinical scenario this algorithm, for all vessel locations, may benefit from further weighting to improve sensitivity. Figure 1 shows a clinical case example with detected LVO.

These triage applications of narrow AI have become a reality within acute stroke workflows and now the discussion of the effectiveness of these solutions is coming to the fore. It’s clear that further improvement is needed to address the challenges around the smaller, more subtle examples of pathologies, which will be tackled by additional training examples and techniques such as ensemble learning.

	All (n = 303)	ICA (n = 160)	MCA M1 (n = 183)	MCA M2 (n = 162)
Accuracy	0.81	0.95	0.89	0.80
Sensitivity	0.73	0.90	0.77	0.51
Specifi city	0.98	0.98	0.98	0.98
Positive predictive value	0.99	0.96	0.97	0.94
Negative predictive value	0.64	0.94	0.84	0.77
F1 score	0.84	0.93	0.86	0.66
Matthews correlation coefficient	0.67	0.89	0.78	0.59

Table 1. Accuracy, sensitivity, specificity, positive predictive value, negative predictive value, F1 score and Matthews correlation metrics corresponding to the LVO detection algorithm for all and each occlusion site.

Reference
Rava et al. | Validation of an Artificial Intelligence Driven Large Vessel Occlusion Detection Algorithm for Acute Ischemic Stroke Patients | The Neuroradiology Journal (2021)
https://pubmed.ncbi.nlm.nih.gov/33657922/

• Rava et al. | Assessment of an Artificial Intelligence Algorithm for Detection of Intracranial Hemorrhage | World Neurosurgery (2021)
  https://pubmed.ncbi.nlm.nih.gov/33684578/
• Rava et al. | Validation of an Artificial Intelligence Driven Large Vessel Occlusion Detection Algorithm for Acute Ischemic Stroke Patients | The Neuroradiology Journal (2021)
  https://pubmed.ncbi.nlm.nih.gov/33657922/
• Rava et al. | Assessment of a Bayesian Vitrea CT Perfusion Analysis to Predict Final Infarct and Penumbra Volumes in Patients with Acute Ischemic Stroke: A Comparison with RAPID | American Journal of Neuroradiology (AJNR) (2020)
  https://pubmed.ncbi.nlm.nih.gov/31948951/
• Rava et al. | Assessment of computed tomography perfusion software in predicting spatial location and volume of infarct in acute ischemic stroke patients: a comparison of Sphere, Vitrea, and RAPID | Journal of NeuroInterventional Surgery (JNIS) (2020)
  https://pubmed.ncbi.nlm.nih.gov/32457224/
• Rava et al. | Enhancing performance of a computed tomography perfusion software for improved prediction of final infarct volume in acute ischemic stroke patients | The Neuroradiology Journal (2021)
  https://pubmed.ncbi.nlm.nih.gov/33472519/
• Ohno et al. | Machine learning for lung CT texture analysis: Improvement of inter-observer agreement for radiological finding classification in patients with pulmonary diseases | Eur J Radiol. (2021)
  https://pubmed.ncbi.nlm.nih.gov/33246272/

• Brestel et al. | RadBot-CXR: Classification of Four Clinical Finding Categories in Chest X-Ray Using Deep Learning | Medicine (2018)
  https://openreview.net/pdf?id=B1tMhcIDM

• Tsuchiya et al. | Deep learning-based image quality improvement of 18F-fluorodeoxyglucose positron emission tomography: a retrospective observational study | EJNMMI Phys. 2021
  https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7994470/

• Á. V. Juanco Müller, J. F. C. Mota, C. Hoogendoorn Segmentation of Skin Lesions by Superpixel Classifi cation with Graph- Context CNN MIUA 2021
  
• Schrempf, P., Watson, H., Park, E., Pajak, M., MacKinnon, H., Muir, K.W., Harris-Birtill, D. and O’Neil, A.Q. (2021). Templated text synthesis for expertguided multi-label extraction from radiology reports. Machine Learning and Knowledge Extraction, 3(2), pp.299-317.
  
• Anderson, O., Kidd, A.C., Goatman, K.A., Weir, A.J., Voisey, J., Dilys, V., Siebert, J.P., and Blyth, K.G. (2020). Fully automated volumetric measurement of malignant pleural mesothelioma from computed tomography images by deep learning: Preliminary results of an internal validation. In Proceedings of the 13th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2020) - Volume 2: BIOIMAGING, pp. 64–73, SCITEPRESS.
  
• Elskhawy, A., Lisowska, A., Keicher, M., Henry, J., Thomson, P. and Navab, N. (2020). Continual Class Incremental Learning for CT Thoracic Segmentation. In MICCAI workshop on “Domain Adaptation and Representation Transfer” (DART).
  
• Jacenków, G., O’Neil, A.Q., Mohr, B. and Tsaftaris, S.A. (2020). INSIDE: Steering Spatial Attention with Non-imaging Information in CNNs. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 385-395). Springer, Cham.
  
• Liu, X., Thermos, S., Chartsias, A., O’Neil, A. and Tsaftaris, S.A. (2021). Disentangled Representations for Domain-Generalized Cardiac Segmentation. In MICCAI workshop on “Statistical Atlases and Computational Models of the Heart” (STACOM), submission to the Multi- Centre, Multi-Vendor & Multi-Disease Cardiac Image Segmentation (M&Ms) Challenge.
  
• Schrempf, P., Watson, H., Mikhael, S., Pajak, M., Falis, M., Lisowska, A., Muir, K.W., Harris-Birtill, D. and O’Neil, A.Q. (2020). Paying Per-Label Attention for Multi-label Extraction from Radiology Reports. In MICCAI workshop on “Large-scale Annotation of Biomedical Data and Expert Label Synthesis” (LABELS).
  
• Shaw, S., Pajak, M., Lisowska, A., Tsaftaris, S.A., O’Neil, A.Q., (2020). Teacher- Student chain for effi cient semi-supervised histology image classifi cation. In ICLR workshop on “AI for Aff ordable Healthcare” (AI4AH).
  
• Appelgren, M., Schrempf, P., Falis, M., Ikeda, S. and O’Neil, A.Q., (2019). Language Transfer for Early Warning of Epidemics from Social Media. In NeurIPS workshop on “Artifi cial Intelligence for Humanitarian Assistance and Disaster Response” (AI+HADR).
  
• Falis, M., Pajak, M., Lisowska, A., Schrempf, P., Deckers, L., Mikhael, S., Tsaftaris, S. and O’Neil, A. Q., (2019). Ontological attention ensembles for capturing semantic concepts in ICD code prediction from clinical text. In EM NLP Workshop “Tenth International Workshop on Health Text Mining and Information Analysis” (LOUHI), p. 168-177.
  
• Jacenków, G., Chartsias, A., Mohr, B. and Tsaftaris, S.A. (2019). Conditioning Convolutional Segmentation Architectures with Non-Imaging Data. Medical Imaging with Deep Learning (Extended Abstract Track).
  
• Zotova, D., Lisowska, A., Anderson, O., Dilys, V. and O’Neil, A. Q., (2019). Comparison of active learning strategies applied to lung nodule segmentation in CT scans. In MICCAI workshop on “Large-scale Annotation of Biomedical Data and Expert Label Synthesis” (LABELS).
  
• Daykin, M., Sellathurai, M. and Poole, I., 2018, July. A Comparison of Unsupervised Abnormality Detection Methods for Interstitial Lung Disease. In Annual Conference on Medical Image Understanding and Analysis (pp. 287-298). Springer, Cham.
  
• Lisowska, A., O’Neil, A. and Poole, I. (2018) Cross-cohort Evaluation of Machine Learning Approaches to Fall Detection from Accelerometer Data. In Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 5: HEALTHINF, January 2018, ISBN 978-989-758-281-3, pages 77-82.
  
• O’Neil, A.Q., Kascenas, A., Henry, J., Wyeth, D., Shepherd, M., Beveridge, E., Clunie, L., Sansom, C., Šeduikytė, E., Muir, K. and Poole, I., (2018). Attaining human-level performance with atlas location autocontext for anatomical landmark detection in 3D CT data. ECCV 2018 Workshop “Geometry Meets Deep Learning”.
  
• Sloan, J., Goatman, K. and Siebert, J. (2018). Learning Rigid Image Registration - Utilizing Convolutional Neural Networks for Medical Image Registration. In Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 2: BIOIMAGING, ISBN 978-989-758-278-3, pages 89-99.
  
• Zhang et al. | A cascaded deep-learning reconstruction method for sparse-view kV-switching dual-energy CT | In Proceedings Volume 11312, Medical Imaging 2020: Physics of Medical Imaging; 1131223 (2020)
  https://doi.org/10.1117/12.2549852
• Cheng Lee et al. | Deep learning based adaptive fi ltering for projection data noise reduction in x-ray computed tomography | In Proceedings Volume 11072, 15th International Meeting on Fully Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine; 110721D (2019)
  https://doi.org/10.1117/12.2534838
• Matsuura et al. | Feature-Aware Deep-Learning Reconstruction for Context-Sensitive X-ray Computed Tomography | In IEEE Transactions on Radiation and Plasma Medical Sciences (Volume: 5, Issue: 1, 2021)
  https://ieeexplore.ieee.org/document/9272677
• Chan et al.| Noise Adaptive Deep Learning Convolution Neural Network for Whole-Body PET Denoising | MIC 2018
  https://ieeexplore.ieee.org/abstract/document/8824303
• Chan et al. | Feature Oriented Deep Convolutional Neural Network for PET Image Denoising | MIC poster 2018
  https://ieeexplore.ieee.org/document/8824592
• Chan et al. | Noise to Noise Ensemble Training for Whole Body PET Denoising | MIC poster 2019
  https://ieeexplore.ieee.org/document/9059779
• Chan et al. | Estimating Ensemble Bias using Bayesian Convolutional Neural Network | MIC poster 2020
  https://ieeexplore.ieee.org/abstract/document/9508035
• Yang et al. | Assessment of Spatial Resolution in PET Images with Deep Convolutional Neural Networks | MIC poster 2020
  https://ieeexplore.ieee.org/document/9508022

• AI Scientific Evidence
• CT
• MRI
• Healthcare IT