Int J Thyroidol.  2023 May;16(1):111-119. 10.11106/ijt.2023.16.1.111.

Artificial Intelligence Computer-Assisted Diagnosis for Thyroid Nodules: Comparison of Diagnostic Performance Using Original and Mobile Ultrasonography Images

• Cho S
• Lee E
• Lee H
• Lee HS
• Yoon JH
• Park VY
• Rho M
• Yoon J
• Kwak JY

• KMID: 2543016
• DOI: http://doi.org/10.11106/ijt.2023.16.1.111

Abstract

Background and Objectives
This study investigated whether an artificial intelligence computer-assisted diagnosis (AI-CAD) software recently developed in our institution named the Severance Artificial intelligence program (SERA) could show similar diagnostic performance for thyroid cancers using ultrasonographic (US) images from a mobile phone (SERA_M) compared to using images directly downloaded from the pictures archive and communication system (PACS) (SERA_P).
Materials and Methods
From October 2019 to December 2019, 259 thyroid nodules from 259 patients were included. SERA was run on original and mobile images to evaluate SERA_P and SERA_M. Nodules were categorized according to the American College of Radiology Thyroid Imaging Reporting and Data System (ACR TI-RADS). To compare diagnostic performance, a logistic regression analysis was conducted using the Generalized Estimating Equation. The area under the curve (AUC) was calculated using the receiver operating characteristic (ROC) curve, and compared using the Delong Method.
Results
There were 40 cancers (15.4%) and 219 benign lesions (84.6%). The AUC and sensitivity of SERA_M (0.82 and 85%, respectively) were not statistically different from SERA_P (0.8 and 75%, respectively) (p=0.526 and p=0.091, respectively). The AUC of radiologists (0.856) was not significantly different compared to SERA_P and SERA_M (p=0.163 and p=0.414, respectively). The sensitivity of radiologists (77.5%) was not statistically different compared to SERA_P and SERA_M (p=0.739 and p=0.361, respectively).
Conclusion
AI-CAD software using pictures taken by a mobile phone showed comparable diagnostic performance with the same software using images directly from PACS.

Keyword

Thyroid nodule; Ultrasonography; Artificial intelligence; Diagnosis; Computer-assisted

Figure

• 

  Fig. 1 Flowchart of study population. AUS: atypia of undermined significance, FLUS: follicular lesion of undetermined significance, FNA: fine needle aspiration

• 

  Fig. 2 Diagram of overall process of image acquisition and diagnosis. ACR TI-RADS: American College of Radiology Thyroid Imaging Reporting and Data System

• 

  Fig. 3 Ultrasonography images and SERA results of a 71-year-old man with pathologically proven papillary thyroid cancer. The lesion was assessed as ACR TI-RADS category 5 by a radiologist. Compared with the original PACS image (A), the picture taken with a mobile phone (B) was blurry and had lower resolution, but the diagnostic results of SERA were both malignant (A, B, respectively). ACR TI-RADS: American College of Radiology Thyroid Imaging Reporting and Data System, PACS: picture archiving and communication system, SER: Severance Artificial intelligence program

• 

  Fig. 4 Ultrasonography images and SERA diagnostic results of a 58-year-old man with a FNA-proven benign nodule. The lesion was assessed as ACR TI-RADS category 3 nodule by a radiologist. Compared with the original PACS image (A), the picture taken with a mobile phone (B) was brighter and had lower resolution, but the diagnostic results of SERA were both benign (A, B, respectively). ACR TI-RADS: American College of Radiology Thyroid Imaging Reporting and Data System, PACS: picture archiving and communication system, SERA: Severance Artificial intelligence program

• 

  Fig. 5 Area under the ROC curve of SERA_P, SERA_M, and radiologists for differentiating thyroid nodules. ROC curve: receiver operating characteristic curve, SERA: Severance Artificial intelligence program, SERA_M: results from SERA using images taken with a mobile phone, SERA_P: results from SERA using images directly downloaded from PACS

Reference

References

1. Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2018; 68(6):394–424. DOI: 10.3322/caac.21492. PMID: 30207593.

2. Choi SH, Kim EK, Kwak JY, Kim MJ, Son EJ. 2010; Interobserver and intraobserver variations in ultrasound assessment of thyroid nodules. Thyroid. 20(2):167–72. DOI: 10.1089/thy.2008.0354. PMID: 19725777.

3. Koh J, Kim SY, Lee HS, Kim EK, Kwak JY, Moon HJ, et al. 2018; Diagnostic performances and interobserver agreement according to observer experience: a comparison study using three guidelines for management of thyroid nodules. Acta Radiol. 59(8):917–23. DOI: 10.1177/0284185117744001. PMID: 29181988.

4. Gurcan MN, Boucheron LE, Can A, Madabhushi A, Rajpoot NM, Yener B. 2009; Histopathological image analysis: a review. IEEE Rev Biomed Eng. 2:147–71. DOI: 10.1109/RBME.2009.2034865. PMID: 20671804. PMCID: PMC2910932.

5. Suzuki K. 2017; Overview of deep learning in medical imaging. Radiol Phys Technol. 10(3):257–73. DOI: 10.1007/s12194-017-0406-5. PMID: 28689314.

6. Litjens G, Sanchez CI, Timofeeva N, Hermsen M, Nagtegaal I, Kovacs I, et al. 2016; Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci Rep. 6:26286. DOI: 10.1038/srep26286. PMID: 27212078. PMCID: PMC4876324.

7. Liang X, Yu J, Liao J, Chen Z. 2020; Convolutional neural network for breast and thyroid nodules diagnosis in ultrasound imaging. Biomed Res Int. 2020:1763803. DOI: 10.1155/2020/1763803. PMID: 32420322. PMCID: PMC7199615.

8. Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, et al. 2017; Dermatologist-level classification of skin cancer with deep neural networks. Nature. 542(7639):115–8. DOI: 10.1038/nature21056. PMID: 28117445. PMCID: PMC8382232.

9. Vandenberghe ME, Scott ML, Scorer PW, Soderberg M, Balcerzak D, Barker C. 2017; Relevance of deep learning to facilitate the diagnosis of HER2 status in breast cancer. Sci Rep. 7:45938. DOI: 10.1038/srep45938. PMID: 28378829. PMCID: PMC5380996.

10. Zhang X, Lee VC, Rong J, Lee JC, Liu F. 2022; Deep convolutional neural networks in thyroid disease detection: a multi-classification comparison by ultrasonography and computed tomography. Comput Methods Programs Biomed. 220:106823. DOI: 10.1016/j.cmpb.2022.106823. PMID: 35489145.

11. Chi J, Walia E, Babyn P, Wang J, Groot G, Eramian M. 2017; Thyroid nodule classification in ultrasound images by fine-tuning deep convolutional neural network. J Digit Imaging. 30(4):477–86. DOI: 10.1007/s10278-017-9997-y. PMID: 28695342. PMCID: PMC5537102.

12. Jin Z, Zhu Y, Zhang S, Xie F, Zhang M, Zhang Y, et al. 2020; Ultrasound computer-aided diagnosis (CAD) based on the Thyroid Imaging Reporting and Data System (TI-RADS) to distinguish benign from malignant thyroid nodules and the diagnostic performance of radiologists with different diagnostic experience. Med Sci Monit. 26:e918452. DOI: 10.12659/MSM.918452.

13. Youn I, Lee E, Yoon JH, Lee HS, Kwon MR, Moon J, et al. 2021; Diagnosing thyroid nodules with atypia of undetermined significance/follicular lesion of undetermined significance cytology with the deep convolutional neural network. Sci Rep. 11(1):20048. DOI: 10.1038/s41598-021-99622-0. PMID: 34625636. PMCID: PMC8501016.

14. Koh J, Lee E, Han K, Kim EK, Son EJ, Sohn YM, et al. 2020; Diagnosis of thyroid nodules on ultrasonography by a deep convolutional neural network. Sci Rep. 10(1):15245. DOI: 10.1038/s41598-020-72270-6. PMID: 32943696. PMCID: PMC7498581.

15. Kwak JY, Han KH, Yoon JH, Moon HJ, Son EJ, Park SH, et al. 2011; Thyroid Imaging Reporting and Data System for US features of nodules: a step in establishing better stratification of cancer risk. Radiology. 260(3):892–9. DOI: 10.1148/radiol.11110206. PMID: 21771959.

16. Quesada-González D, Merkoçi A. 2017; Mobile phone-based biosensing: an emerging "diagnostic and communication" technology. Biosens Bioelectron. 92:549–62. DOI: 10.1016/j.bios.2016.10.062.

17. Coleman B, Coarsey C, Asghar W. 2019; Cell phone based colorimetric analysis for point-of-care settings. Analyst. 144(6):1935–47. DOI: 10.1039/C8AN02521E. PMID: 30688961. PMCID: PMC6411445.

18. Tessler FN, Middleton WD, Grant EG, Hoang JK, Berland LL, Teefey SA, et al. 2017; ACR Thyroid Imaging, Reporting and Data System (TI-RADS): white paper of the ACR TI-RADS Committee. J Am Coll Radiol. 14(5):587–95. DOI: 10.1016/j.jacr.2017.01.046. PMID: 28372962.

19. Fujimoto Y, Oka A, Omoto R, Hirose M. 1967; Ultrasound scanning of the thyroid gland as a new diagnostic approach. Ultrasonics. 5:177–80. DOI: 10.1016/S0041-624X(67)80065-9. PMID: 6053923.

20. Gallo M, Pesenti M, Valcavi R. 2003; Ultrasound thyroid nodule measurements: the "gold standard" and its limitations in clinical decision making. Endocr Pract. 9(3):194–9. DOI: 10.4158/ep.9.3.194. PMID: 12917060.

21. Kim GR, Lee E, Kim HR, Yoon JH, Park VY, Kwak JY. 2021; Convolutional neural network to stratify the malignancy risk of thyroid nodules: diagnostic performance compared with the american college of radiology Thyroid Imaging Reporting and Data System implemented by experienced radiologists. AJNR Am J Neuroradiol. 42(8):1513–9. DOI: 10.3174/ajnr.A7149. PMID: 33985947. PMCID: PMC8367605.

22. Choi YJ, Baek JH, Park HS, Shim WH, Kim TY, Shong YK, et al. 2017; A computer-aided diagnosis system using artificial intelligence for the diagnosis and characterization of thyroid nodules on ultrasound: initial clinical assessment. Thyroid. 27(4):546–52. DOI: 10.1089/thy.2016.0372. PMID: 28071987.

23. Fawzy HED. 2015; The accuracy of mobile phone camera instead of high resolution camera in digital close range photogrammetry. Int J Civ Eng Technol. 6(1):76–85.

24. Blaivas M, Lyon M, Duggal S. 2005; Ultrasound image transmission via camera phones for overreading. Am J Emerg Med. 23(4):433–8. DOI: 10.1016/j.ajem.2004.09.037. PMID: 16032606.

25. Goost H, Witten J, Heck A, Hadizadeh DR, Weber O, Graff I, et al. 2012; Image and diagnosis quality of X-ray image transmission via cell phone camera: a project study evaluating quality and reliability. PLoS One. 7(10):e43402. DOI: 10.1371/journal.pone.0043402. PMID: 23082108. PMCID: PMC3474770.

26. Breslauer DN, Maamari RN, Switz NA, Lam WA, Fletcher DA. 2009; Mobile phone based clinical microscopy for global health applications. PLoS One. 4(7):e6320. DOI: 10.1371/journal.pone.0006320. PMID: 19623251. PMCID: PMC2709430.

27. Luo LB, Chong JW. 2012; Real‐time digital image stabilization for cell phone cameras in low‐light environments without frame memory. ETRI Journal. 34(1):138–41. DOI: 10.4218/etrij.12.0211.0338.

28. Gampala V, Kumar MS, Sushama C, Raj EFI. 2020. Deep learning based image processing approaches for image deblurring. Proceedings;Materials Today:

29. Weinreich M, Chudow JJ, Weinreich B, Krumerman T, Nag T, Rahgozar K, et al. 2019; Development of an artificially intelligent mobile phone application to identify cardiac devices on chest radiography. JACC Clin Electrophysiol. 5(9):1094–5. DOI: 10.1016/j.jacep.2019.05.013. PMID: 31537342.

30. Cibas ES, Ali SZ. 2017; The 2017 Bethesda system for reporting thyroid cytopathology. Thyroid. 27(11):1341–6. DOI: 10.1089/thy.2017.0500. PMID: 29091573.