Ultrasonography.  2023 Oct;42(4):518-531. 10.14366/usg.23055.

Diagnostic performance of ultrasound risk stratification systems on thyroid nodules cytologically classified as indeterminate: a systematic review and meta-analysis

• Xing Z
• Qiu Y
• Zhu J
• Su A
• Wu W

• KMID: 2546450
• DOI: http://doi.org/10.14366/usg.23055

Abstract

Purpose
Ultrasound (US) risk stratification systems (RSSs) are increasingly being utilized for the optimal management of thyroid nodules, including those with indeterminate cytology. The goal of this study was to evaluate the category-based diagnostic performance of US RSSs in identifying malignancy in indeterminate nodules.
Methods
This systematic review and meta-analysis was registered on PROSPERO (CRD42021266195). PubMed, EMBASE, and Web of Science were searched through December 1, 2022. Original articles reporting data on the performance of US RSSs for indeterminate nodules were included. The numbers of nodules classified as true negative, true positive, false negative, and false positive were extracted.
Results
Thirty-three studies evaluating 7,225 indeterminate thyroid nodules were included. The diagnostic accuracy was quantitatively synthesized using a Bayesian bivariate model based on the integrated nested Laplace approximation in R. For the intermediate- to high-risk category, the sensitivity levels of the American College of Radiology, the American Thyroid Association, the European Thyroid Association, the Korean Thyroid Association/Korean Society of Thyroid Radiology, and Kwak et al. were found to be 0.80, 0.72, 0.76, 0.96, and 0.97, respectively. The corresponding specificity measurements were 0.36, 0.50, 0.49, 0.28, and 0.17. Furthermore, for the high-risk category, the sensitivity values were 0.40, 0.46, 0.55, 0.47, and 0.10, while the specificity levels were 0.91, 0.90, 0.71, 0.91, and 0.99, respectively.
Conclusion
The overall diagnostic performance of the US RSSs was moderate in the differentiation of indeterminate nodules.

Keyword

Ultrasonography; Thyroid; Biopsy; Diagnosis; Meta-analysis

Figure

• 

  Fig. 1. PRISMA 2020 flow diagram.

• 

  Fig. 2. Quality assessment based on the QUADAS-2 tool [6,21-51].

• 

  Fig. 3. Estimates of ultrasound risk stratification systems for categorization of intermediate to high risk as positive.CrI, credible interval; ACR, American College of Radiology; ATA, American Thyroid Association; EU, European Thyroid Association; K, Korean Society of Thyroid Radiology; LR+, positive likelihood ratio; LR-, negative likelihood ratio; DOR, diagnostic odds ratio; AUC, area under the receiver operating characteristic curve.

• 

  Fig. 4. Estimates of ultrasound risk stratification systems for categorization of high risk as positive.CrI, credible interval; ACR, American College of Radiology; ATA, American Thyroid Association; EU, European Thyroid Association; K, Korean Society of Thyroid Radiology; LR+, positive likelihood ratio; LR-, negative likelihood ratio; DOR, diagnostic odds ratio; AUC, area under the receiver operating characteristic curve.

Reference

References

1. Russ G, Royer B, Bigorgne C, Rouxel A, Bienvenu-Perrard M, Leenhardt L. Prospective evaluation of thyroid imaging reporting and data system on 4550 nodules with and without elastography. Eur J Endocrinol. 2013; 168:649–655.

2. Cibas ES, Ali SZ. The Bethesda System for Reporting Thyroid Cytopathology. Thyroid. 2009; 19:1159–1165.

3. Bongiovanni M, Spitale A, Faquin WC, Mazzucchelli L, Baloch ZW. The Bethesda System for Reporting Thyroid Cytopathology: a meta-analysis. Acta Cytol. 2012; 56:333–339.

4. Haugen BR, Alexander EK, Bible KC, Doherty GM, Mandel SJ, Nikiforov YE, et al. 2015 American Thyroid Association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: the American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer. Thyroid. 2016; 26:1–133.

5. Alexander EK, Faquin WC, Krane JF. Highlights for the cytology community from the 2015 American Thyroid Association clinical guidelines on the management of thyroid nodules and well-differentiated thyroid cancer. Cancer Cytopathol. 2016; 124:453–456.

6. Maia FF, Matos PS, Pavin EJ, Zantut-Wittmann DE. Thyroid imaging reporting and data system score combined with Bethesda system for malignancy risk stratification in thyroid nodules with indeterminate results on cytology. Clin Endocrinol (Oxf). 2015; 82:439–444.

7. Grani G, Lamartina L, Ascoli V, Bosco D, Nardi F, D'Ambrosio F, et al. Ultrasonography scoring systems can rule out malignancy in cytologically indeterminate thyroid nodules. Endocrine. 2017; 57:256–261.

8. Migda B, Migda M, Migda MS. A systematic review and meta-analysis of the Kwak TIRADS for the diagnostic assessment of indeterminate thyroid nodules. Clin Radiol. 2019; 74:123–130.

9. Horvath E, Majlis S, Rossi R, Franco C, Niedmann JP, Castro A, et al. An ultrasonogram reporting system for thyroid nodules stratifying cancer risk for clinical management. J Clin Endocrinol Metab. 2009; 94:1748–1751.

10. Sahli ZT, Karipineni F, Hang JF, Canner JK, Mathur A, Prescott JD, et al. The association between the ultrasonography TIRADS classification system and surgical pathology among indeterminate thyroid nodules. Surgery. 2019; 165:69–74.

11. McInnes MDF, Moher D, Thombs BD, McGrath TA, Bossuyt PM; PRISMA-DTA Group, et al. Preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies: the PRISMA-DTA statement. JAMA. 2018; 319:388–396.

12. Gharib H, Papini E, Garber JR, Duick DS, Harrell RM, Hegedus L, et al. American Association of Clinical Endocrinologists, American College of Endocrinology, and Associazione Medici Endocrinologi medical guidelines for clinical practice for the diagnosis and management of thyroid nodules--2016 update. Endocr Pract. 2016; 22:622–639.

13. Tessler FN, Middleton WD, Grant EG, Hoang JK, Berland LL, Teefey SA, et al. ACR Thyroid Imaging, Reporting and Data System (TI-RADS): white paper of the ACR TI-RADS Committee. J Am Coll Radiol. 2017; 14:587–595.

14. Perros P, Boelaert K, Colley S, Evans C, Evans RM, Gerrard Ba G, et al. Guidelines for the management of thyroid cancer. Clin Endocrinol (Oxf). 2014; 81 Suppl 1:1–122.

15. Russ G, Bonnema SJ, Erdogan MF, Durante C, Ngu R, Leenhardt L. European Thyroid Association guidelines for ultrasound malignancy risk stratification of thyroid nodules in adults: the EU-TIRADS. Eur Thyroid J. 2017; 6:225–237.

16. Russ G. Risk stratification of thyroid nodules on ultrasonography with the French TI-RADS: description and reflections. Ultrasonography. 2016; 35:25–38.

17. Shin JH, Baek JH, Chung J, Ha EJ, Kim JH, Lee YH, et al. Ultrasonography diagnosis and imaging-based management of thyroid nodules: revised Korean Society of Thyroid Radiology consensus statement and recommendations. Korean J Radiol. 2016; 17:370–395.

18. Kwak JY, Han KH, Yoon JH, Moon HJ, Son EJ, Park SH, et al. Thyroid imaging reporting and data system for US features of nodules: a step in establishing better stratification of cancer risk. Radiology. 2011; 260:892–899.

19. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011; 155:529–536.

20. Guo J, Riebler A, Rue H. Bayesian bivariate meta-analysis of diagnostic test studies with interpretable priors. Stat Med. 2017; 36:3039–3058.

21. Ahmadi S, Herbst R, Oyekunle T, Jiang X, Strickland K, Roman S, et al. Using the ATA and ACR TI-RADS sonographic classifications as adjunctive predictors of malignancy for indeterminate thyroid nodules. Endocr Pract. 2019; 25:908–917.

22. Al Dawish M, Alwin Robert A, Al Shehri K, Hawsawi S, Mujammami M, Al Basha IA, et al. Risk stratification of thyroid nodules with Bethesda III category: the experience of a territorial healthcare hospital. Cureus. 2020; 12:e8202.

23. Barbosa TL, Mesa Junior CO, Graf H, Cavalvanti T, Trippia MA, da Silveira Ugino RT, et al. ACR TI-RADS and ATA US scores are helpful for the management of thyroid nodules with indeterminate cytology. BMC Endocr Disord. 2019; 19:112.

24. Baser H, Cakir B, Topaloglu O, Alkan A, Polat SB, Dogan HT, et al. Diagnostic accuracy of Thyroid Imaging Reporting and Data System in the prediction of malignancy in nodules with atypia and follicular lesion of undetermined significance cytologies. Clin Endocrinol (Oxf). 2017; 86:584–590.

25. Capezzone M, Cantara S, Di Santo A, Sagnella A, Pilli T, Brilli L, et al. The combination of sonographic features and the seven-gene panel may be useful in the management of thyroid nodules with indeterminate cytology. Front Endocrinol (Lausanne). 2021; 12:613727.

26. Celletti I, Fresilli D, De Vito C, Bononi M, Cardaccio S, Cozzolino A, et al. TIRADS, SRE and SWE in INDETERMINATE thyroid nodule characterization: which has better diagnostic performance? Radiol Med. 2021; 126:1189–1200.

27. Chaigneau E, Russ G, Royer B, Bigorgne C, Bienvenu-Perrard M, Rouxel A, et al. TIRADS score is of limited clinical value for risk stratification of indeterminate cytological results. Eur J Endocrinol. 2018; 179:13–20.

28. Chirayath SR, Pavithran PV, Abraham N, Nair V, Bhavani N, Kumar H, et al. Prospective study of Bethesda categories III and IV thyroid nodules: outcomes and predictive value of BRAF(V600E) mutation. Indian J Endocrinol Metab. 2019; 23:278–281.

29. He YP, Xu HX, Zhao CK, Sun LP, Li XL, Yue WW, et al. Cytologically indeterminate thyroid nodules: increased diagnostic performance with combination of US TI-RADS and a new scoring system. Sci Rep. 2017; 7:6906.

30. Hong MJ, Na DG, Baek JH, Sung JY, Kim JH. Cytologyultrasonography risk-stratification scoring system based on fine-needle aspiration cytology and the Korean-Thyroid Imaging Reporting and Data System. Thyroid. 2017; 27:953–959.

31. Hong HS, Lee JY. Diagnostic performance of ultrasound patterns by K-TIRADS and 2015 ATA guidelines in risk stratification of thyroid nodules and follicular lesions of undetermined significance. AJR Am J Roentgenol. 2019; 213:444–450.

32. Slowinska-Klencka D, Wysocka-Konieczna K, Klencki M, Popowicz B. Diagnostic value of six thyroid imaging reporting and data systems (TIRADS) in cytologically equivocal thyroid nodules. J Clin Med. 2020; 9:2281.

33. Koh J, Kim EK, Kwak JY, Yoon JH, Moon HJ. Repeat fine-needle aspiration can be performed at 6 months or more after initial atypia of undetermined significance or follicular lesion of undetermined significance results for thyroid nodules 10 mm or larger. Eur Radiol. 2016; 26:4442–4448.

34. Mao F, Xu HX, Zhao CK, Bo XW, Li XL, Li DD, et al. Thyroid imaging reporting and data system in assessment of cytological Bethesda Category III thyroid nodules. Clin Hemorheol Microcirc. 2017; 65:163–173.

35. Marina M, Zatelli MC, Goldoni M, Del Rio P, Corcione L, Martorana D, et al. Combination of ultrasound and molecular testing in malignancy risk estimate of Bethesda category IV thyroid nodules: results from a single-institution prospective study. J Endocrinol Invest. 2021; 44:2635–2643.

36. Mehta S, Kannan S. Approaching indeterminate thyroid nodules in the absence of molecular markers: "The BETH-TR score". Indian J Endocrinol Metab. 2020; 24:170–175.

37. Park VY, Kim EK, Kwak JY, Yoon JH, Moon HJ. Malignancy risk and characteristics of thyroid nodules with two consecutive results of atypia of undetermined significance or follicular lesion of undetermined significance on cytology. Eur Radiol. 2015; 25:2601–2607.

38. Piccardo A, Puntoni M, Dezzana M, Bottoni G, Foppiani L, Marugo A, et al. Indeterminate thyroid nodules: the role of 18F-FDG PET/CT in the "era" of ultrasonography risk stratification systems and new thyroid cytology classifications. Endocrine. 2020; 69:553–561.

39. Rho M, Kim EK, Moon HJ, Yoon JH, Park VY, Han K, et al. Clinical parameter for deciding the BRAFV600E mutation test in atypia of undetermined significance/follicular lesion of undetermined significance thyroid nodules: US features according to TIRADS. Ultrasound Q. 2017; 33:284–288.

40. Rocha TG, Rosario PW, Silva AL, Nunes MB, Calsolari MR. Thyroid Imaging Reporting and Data System (TI-RADS) of the American College of Radiology (ACR) for predicting malignancy in thyroid nodules >1 cm with indeterminate cytology. Diagn Cytopathol. 2019; 47:523–525.

41. Rosario PW, Silva AL, Calsolari MR. The ATA classification and TI-RADS ACR predict not only benignity but also the histology of nonbenign tumors in thyroid nodules with indeterminate cytology. Diagn Cytopathol. 2021; 49:165–167.

42. Sahli ZT, Sharma AK, Canner JK, Karipineni F, Ali O, Kawamoto S, et al. TIRADS interobserver variability among indeterminate thyroid nodules: a single-institution study. J Ultrasound Med. 2019; 38:1807–1813.

43. Suh YJ, Choi YJ. Strategy to reduce unnecessary surgeries in thyroid nodules with cytology of Bethesda category III (AUS/FLUS): a retrospective analysis of 667 patients diagnosed by surgery. Endocrine. 2020; 69:578–586.

44. Sultan R, Levy S, Sulanc E, Honasoge M, Rao SD. Utility of afirma gene expression classifier for evaluation of indeterminate thyroid nodules and correlation with ultrasound risk assessment: single institutional experience. Endocr Pract. 2020; 26:543–551.

45. Trimboli P, Fulciniti F, Zilioli V, Ceriani L, Giovanella L. Accuracy of international ultrasound risk stratification systems in thyroid lesions cytologically classified as indeterminate. Diagn Cytopathol. 2017; 45:113–117.

46. Ulisse S, Bosco D, Nardi F, Nesca A, D'Armiento E, Guglielmino V, et al. Thyroid Imaging Reporting and Data System score combined with the new Italian classification for thyroid cytology improves the clinical management of indeterminate nodules. Int J Endocrinol. 2017; 2017:9692304.

47. Wang MM, Beckett K, Douek M, Masamed R, Patel M, Tseng CH, et al. Diagnostic value of molecular testing in sonographically suspicious thyroid nodules. J Endocr Soc. 2020; 4:bvaa081.

48. Wu H, Zhang B, Cai G, Li J, Gu X. American College of Radiology Thyroid Imaging Report and Data System combined with K-RAS mutation improves the management of cytologically indeterminate thyroid nodules. PLoS One. 2019; 14:e0219383.

49. Yoo WS, Ahn HY, Ahn HS, Chung YJ, Kim HS, Cho BY, et al. Malignancy rate of Bethesda category III thyroid nodules according to ultrasound risk stratification system and cytological subtype. Medicine (Baltimore). 2020; 99:e18780.

50. Yoon JH, Kwon HJ, Kim EK, Moon HJ, Kwak JY. Subcategorization of atypia of undetermined significance/follicular lesion of undetermined significance (AUS/FLUS): a study applying Thyroid Imaging Reporting and Data System (TIRADS). Clin Endocrinol (Oxf). 2016; 85:275–282.

51. Zhang WB, Li JJ, Chen XY, He BL, Shen RH, Liu H, et al. SWE combined with ACR TI-RADS categories for malignancy risk stratification of thyroid nodules with indeterminate FNA cytology. Clin Hemorheol Microcirc. 2020; 76:381–390.

52. Kim DH, Chung SR, Choi SH, Kim KW. Accuracy of Thyroid Imaging Reporting and Data System category 4 or 5 for diagnosing malignancy: a systematic review and meta-analysis. Eur Radiol. 2020; 30:5611–5624.

53. Kim DH, Kim SW, Basurrah MA, Lee J, Hwang SH. Diagnostic performance of six ultrasound risk stratification systems for thyroid nodules: a systematic review and network meta-analysis. AJR Am J Roentgenol. 2023; 220:791–803.

54. Grani G, Lamartina L, Cantisani V, Maranghi M, Lucia P, Durante C. Interobserver agreement of various thyroid imaging reporting and data systems. Endocr Connect. 2018; 7:1–7.

55. Sych YP, Fadeev VV, Fisenko EP, Kalashnikova M. Reproducibility and interobserver agreement of different Thyroid Imaging and Reporting Data Systems (TIRADS). Eur Thyroid J. 2021; 10:161–167.

56. Kim PH, Suh CH, Baek JH, Chung SR, Choi YJ, Lee JH. Diagnostic performance of four ultrasound risk stratification systems: a systematic review and meta-analysis. Thyroid. 2020; 30:1159–1168.

57. Chen JC, Pace SC, Chen BA, Khiyami A, McHenry CR. Yield of repeat fine-needle aspiration biopsy and rate of malignancy in patients with atypia or follicular lesion of undetermined significance: the impact of the Bethesda System for Reporting Thyroid Cytopathology. Surgery. 2012; 152:1037–1044.

58. Dincer N, Balci S, Yazgan A, Guney G, Ersoy R, Cakir B, et al. Follow-up of atypia and follicular lesions of undetermined significance in thyroid fine needle aspiration cytology. Cytopathology. 2013; 24:385–390.

59. McElroy MK, Mahooti S, Hasteh F. A single institution experience with the new Bethesda System for Reporting Thyroid Cytopathology: correlation with existing cytologic, clinical, and histological data. Diagn Cytopathol. 2014; 42:564–569.

60. Nayar R, Ivanovic M. The indeterminate thyroid fine-needle aspiration: experience from an academic center using terminology similar to that proposed in the 2007 National Cancer Institute Thyroid Fine Needle Aspiration State of the Science Conference. Cancer. 2009; 117:195–202.