Hanyang Med Rev.  2017 Nov;37(2):61-70. 10.7599/hmr.2017.37.2.61.

Deep Learning for Medical Image Analysis: Applications to Computed Tomography and Magnetic Resonance Imaging

Affiliations
  • 1VUNO Inc., Seoul, Korea. kyuhwanjung@gmail.com
  • 2School of Medicine, Imperial College London, London, United Kingdom.

Abstract

Recent advances in deep learning have brought many breakthroughs in medical image analysis by providing more robust and consistent tools for the detection, classification and quantification of patterns in medical images. Specifically, analysis of advanced modalities such as computed tomography (CT) and magnetic resonance imaging (MRI) has benefited most from the data-driven nature of deep learning. This is because the need of knowledge and experience-oriented feature engineering process can be circumvented by automatically deriving representative features from the complex high dimensional medical images with respect to the target tasks. In this paper, we will review recent applications of deep learning in the analysis of CT and MR images in a range of tasks and target organs. While most applications are focused on the enhancement of the productivity and accuracy of current diagnostic analysis, we will also introduce some promising applications which will significantly change the current workflow of medical imaging. We will conclude by discussing opportunities and challenges of applying deep learning to advanced imaging and suggest future directions in this domain.

Keyword

Deep Learning; Medical Imaging; Computed Tomography; Magnetic Resonance Imaging

MeSH Terms

Classification
Diagnostic Imaging
Efficiency
Learning*
Magnetic Resonance Imaging*

Figure

  • Fig. 1A Visualization of ‘evidence hotspot’ in spinal MRI [10]. Adapted with permission

  • Fig. 1B HIV patient and normal control HIV patient and healthy control

  • Fig. 2 Multi-stream, multi-scale CNN architecture for pulmonary nodule classification [16]. Adapted with permission.

  • Fig. 3 Segmentation of aggressive prostate cancer lesion using GAN [29]. Adapted with permission.

  • Fig. 4 Conversion of MRI to CT using GAN [38]. Adapted with permission

  • Fig. 5 Example of a model adopting deep learning for survival analysis [40]. Adapted with permission.


Cited by  1 articles

Artificial Intelligence in Medicine
Jihoon Jeong
Hanyang Med Rev. 2017;37(2):47-48.    doi: 10.7599/hmr.2017.37.2.47.


Reference

1. Litjens G, Kooi T, Bejnordi BE, et al. A Survey on Deep Learning in Medical Image Analysis. arXiv [Internet]. 2017. 1702.05747:1-34. Available from: http://arxiv.org/abs/1702.05747/.
2. Lot SB, Lint JJ, Freedmant MT, Munt SK. Computer-Assisted Diagnosis of Lung Nodule Detection Using Artificial Convolution Neural Network. Med Imaging. 1993; 1898:859–869.
3. Chan HP, Lo SC, Sahiner B, Lam KL, Helvie MA. Computer-aided detection of mammographic microcalcifications: pattern recognition with an artificial neural network. Med Phys. 1995; 22:1555–1567.
Article
4. Imagenet Challenge. Available from: http://image-net.org/challenges/LSVRC/2012/results.html.
5. Ciresan DC, Giusti A, Gambardella LM, Schmidhuber J. Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks. Med Image Comput Comput Assist Interv. 2013; 16:411–418.
6. Gulshan V, Peng L, Coram M, et al. Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. JAMA. 2016; 316:2402–2410.
Article
7. Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017; 542:115–118.
Article
8. Hosseini-Asl E, Keynton R, El-Baz A. Alzheimer's disease diagnostics by adaptation of 3D convolutional network. ICIP. 2016; 126–130.
Article
9. Gao M, Xu Z, Lu L, Harrison AP, Summers RM, Mollura DJ. Holistic Interstitial Lung Disease Detection using Deep Convolutional Neural Networks: Multi-label Learning and Unordered Pooling. arXiv [Internet]. 2017. 9352:1-9. Available from: http://arxiv.org/abs/1701.05616/.
10. Jamaludin A, Kadir T, Zisserman A. SpineNet: Automatically pinpointing classification evidence in spinal MRIs. Lect Notes Comput Sci. 2016; 9901:166–175.
Article
11. Zintgraf L, Cohen T, Adel T, Welling M. Visualizing Deep Neural Network Decisions: Prediction Difference Analysis. arXiv [Internet]. 2016. 1511.06488:1-12. Available from: http://arxiv.org/abs/1511.06488/.
12. Zeiler MD, Fergus R. Visualizing and Understanding Convolutional Networks. Comput Vis ECCV. 2014; 8689:818–833.
Article
13. Cheng JZ, Ni D, Chou YH, et al. Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans. Sci Rep. 2016; 6:24454.
Article
14. Shin HC, Roth HR, Gao M, et al. Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning. IEEE Trans Med Imaging. 2016; 35:1285–1298.
Article
15. Dou Q, Chen H, Yu L, Qin J, Heng PA. Multi-level Contextual 3D CNNs for False Positive Reduction in Pulmonary Nodule Detection. IEEE Trans Biomed Eng. 2017; 64:1558–1567.
Article
16. Ciompi F, Chung K, van Riel SJ, et al. Towards automatic pulmonary nodule management in lung cancer screening with deep learning. Sci Rep. 2017; 7:46479.
Article
17. Ghafoorian M, Karssemeijer N, Heskes T, et al. Deep multi-scale location-aware 3D convolutional neural networks for automated detection of lacunes of presumed vascular origin. NeuroImage Clin. 2017; 14:391–399.
Article
18. Guo Y, Wu G, Commander LA, et al. Segmenting hippocampus from infant brains by sparse patch matching with deep-learned features. Med Image Comput Comput Assist Interv. 2014; 17:308–315.
Article
19. Lo P, Van Ginneken B, Reinhardt JM, et al. Extraction of airways from CT (EXACT’09). IEEE Trans Med Imaging. 2012; 31:2093–2107.
Article
20. Charbonnier JP, Rikxoort EMV, Setio AAA, Schaefer-Prokop CM, Ginneken BV, Ciompi F. Improving airway segmentation in computed tomography using leak detection with convolutional networks. Med Image Anal. 2017; 36:52–60.
Article
21. Hu P, Wu F, Peng J, Liang P, Kong D. Automatic 3D liver segmentation based on deep learning and globally optimized surface evolution. Phys Med Biol. 2016; 61(24):8676–8698.
Article
22. Thong W, Kadoury S, Piché N, Pal CJ. Convolutional networks for kidney segmentation in contrast-enhanced CT scans. Comput Methods Biomech Biomed Eng Imaging Vis. 2016; 1163:1–6.
Article
23. Roth HR, Lu L, Farag A, Sohn A, Summers RM. Spatial aggregation of holistically-nested networks for automated pancreas segmentation. Lect Notes Comput Sci. 2016; 9901:451–459.
Article
24. Yu L, Yang X, Chen H, Qin J, Heng P-A. Volumetric ConvNets with Mixed Residual Connections for Automated Prostate Segmentation from 3D MR Images. In : Thirty-First AAAI Conference on Artificial Intelligence; 2017. p. 66–72.
25. Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. MICCAI. 2015; 234–241.
Article
26. Poudel RPK, Lamata P, Montana G. Recurrent fully convolutional neural networks for multi-slice MRI cardiac segmentation. Lect Notes Comput Sci. 2017; 10129:83–94.
Article
27. Brosch T, Tang LYW, Yoo Y, Li DKB, Traboulsee A, Tam R. Deep 3D Convolutional Encoder Networks With Shortcuts for Multiscale Feature Integration Applied to Multiple Sclerosis Lesion Segmentation. IEEE Trans Med Imaging. 2016; 35(5):1229–1239.
Article
28. Ghafoorian M, Karssemeijer N, Heskes T, et al. Location Sensitive Deep Convolutional Neural Networks for Segmentation of White Matter Hyperintensities. Sci Rep. 2017; 7:5110.
Article
29. Kohl S, Bonekamp D, Schlemmer H-P, Yaqubi K, Hohenfellner M, Hadaschik B, et al. Adversarial Networks for the Detection of Aggressive Prostate Cancer. arXiv [Internet]. 2017. 1702.08014:1-12. Available from: http://arxiv.org/abs/1702.08014/.
30. Cheng X, Zhang L, Zheng Y. Deep similarity learning for multimodal medical images. Comput Methods Biomech Biomed Eng Imaging Vis. 2016; 1163:1–5.
Article
31. Simonovsky M, Gutiérrez-Becker B, Mateus D, Navab N, Komodakis N. A Deep Metric for Multimodal Registration. arXiv [Internet]. 2016. 1609.05396:1-10. Available from: http://arxiv.org/abs/1609.05396/.
32. Miao S, Wang ZJ, Liao R. A CNN Regression Approach for Real-Time 2D/3D Registration. IEEE Trans Med Imaging. 2016; 35(5):1352–1363.
Article
33. Yang X, Kwitt R, Styner M, Niethammer M. Fast Predictive Image Registration. arXiv [Internet]. 2017. 1703.10902:1-10. Available from: http://arxiv.org/abs/1703.10902/.
34. Chen H, Zhang Y, Kalra MK, et al. Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN). IEEE Trans Med Imaging. 2017; 36:2524–2535.
35. Bahrami K, Shi F, Rekik I, Shen D. Convolutional Neural Network for Reconstruction of 7T-like Images from 3T MRI Using Appearance and Anatomical Features. In : MICCAI 2016 DL workshop; 2016. p. 39–47.
36. Oktay O, Bai W, Lee M, Guerrero R, Kamnitsas K, Caballero J, et al. Multiinput cardiac image super-resolution using convolutional neural networks. Lect Notes Comput Sci. 2016; 9902:246–254.
Article
37. Li R, Zhang W, Suk H-I, et al. Deep learning based imaging data completion for improved brain disease diagnosis. Med Image Comput Comput Assist Interv. 2014; 17:305–312.
Article
38. Nie D, Trullo R, Petitjean C, Ruan S, Shen D. Medical Image Synthesis with Context-Aware Generative Adversarial Networks. arXiv [Internet]. 2016. 1612.05362:1-11. Available from: http://arxiv.org/abs/1612.05362/.
39. Nie D, Zhang H, Adeli E, Liu L, Shen D. 3D deep learning for multi-modal imaging-guided survival time prediction of brain tumor patients. Med Image Comput Comput Assist Interv. Med Image Comput Comput Assist Interv. 2016; 9901:212–220.
40. Van der Burgh HK, Schmidt R, Westeneng HJ, de Reus MA, de Reus MA, van den Berg LH. Deep learning predictions of survival based on MRI in amyotrophic lateral sclerosis. NeuroImage Clin. 2017; 13:361–369.
Article
41. Yoo Y, Tang LW, Brosch T, Li DKB, Metz L, Traboulsee A, Tam R. Deep learning of brain lesion patterns for predicting future disease activity in patients with early symptoms of multiple sclerosis. Deep Learn Data Label Med Appl (2016). Lect Notes Comput Sci. 2016; 10008:86–94.
42. Oakden-Rayner L, Carneiro G, Bessen T, Nascimento JC, Bradley AP, Palmer LJ. Precision Radiology: Predicting longevity using feature engineering and deep learning methods in a radiomics framework. Sci Rep. 2017; 7:1648.
Article
43. Leibig C, Allken V, Berens P, Wahl S. Leveraging uncertainty information from deep neural networks for disease detection. bioRxiv. 2016; 084210:1–21. Available from: http://www.biorxiv.org/content/early/2016/10/28/084210/.
Article
44. Burak Akgül C, Rubin DL, Napel S, Beaulieu CF, Greenspan H, Acar B. Content-based image retrieval in radiology: Current status and future directions. J Digit Imaging. 2011; 24:208–222.
Article
45. Shokri R, Shmatikov V. Privacy-preserving deep learning. In : 53rd Annual Allerton Conference on Communication, Control, and Computing, Allerton; 2015. p. 909–910.
46. Papernot N, Abadi MM, Erlingsson Ú, Goodfellow I, Talwar K. Semisupervised knowledge transfer for deep learning from private training data. arXiv [Internet]. 2016. 1610.05755:1-16. Available from: http://arxiv.org/abs/1610.05755/.
Full Text Links
  • HMR
Actions
Cited
CITED
export Copy
Close
Share
  • Twitter
  • Facebook
Similar articles
Copyright © 2024 by Korean Association of Medical Journal Editors. All rights reserved.     E-mail: koreamed@kamje.or.kr