Korean J Radiol.  2016 Oct;17(5):706-714. 10.3348/kjr.2016.17.5.706.

Does the Reporting Quality of Diagnostic Test Accuracy Studies, as Defined by STARD 2015, Affect Citation?

• Choi YJ
• Chung MS
• Koo HJ
• Park JE
• Yoon HM
• Park SH

• KMID: 2458062
• DOI: http://doi.org/10.3348/kjr.2016.17.5.706

Abstract

OBJECTIVE
To determine the rate with which diagnostic test accuracy studies that are published in a general radiology journal adhere to the Standards for Reporting of Diagnostic Accuracy Studies (STARD) 2015, and to explore the relationship between adherence rate and citation rate while avoiding confounding by journal factors.
MATERIALS AND METHODS
All eligible diagnostic test accuracy studies that were published in the Korean Journal of Radiology in 2011-2015 were identified. Five reviewers assessed each article for yes/no compliance with 27 of the 30 STARD 2015 checklist items (items 28, 29, and 30 were excluded). The total STARD score (number of fulfilled STARD items) was calculated. The score of the 15 STARD items that related directly to the Quality Assessment of Diagnostic Accuracy Studies (QUADAS)-2 was also calculated. The number of times each article was cited (as indicated by the Web of Science) after publication until March 2016 and the article exposure time (time in months between publication and March 2016) were extracted.
RESULTS
Sixty-three articles were analyzed. The mean (range) total and QUADAS-2-related STARD scores were 20.0 (14.5-25) and 11.4 (7-15), respectively. The mean citation number was 4 (0-21). Citation number did not associate significantly with either STARD score after accounting for exposure time (total score: correlation coefficient = 0.154, p = 0.232; QUADAS-2-related score: correlation coefficient = 0.143, p = 0.266).
CONCLUSION
The degree of adherence to STARD 2015 was moderate for this journal, indicating that there is room for improvement. When adjusted for exposure time, the degree of adherence did not affect the citation rate.

Keyword

STARD; STARD 2015; Citation; Reporting quality; Accuracy; Adherence; Impact; Impact factor

MeSH Terms

Diagnostic Tests, Routine/*standards
Guideline Adherence/statistics & numerical data
Guidelines as Topic
Humans
*Journal Impact Factor
Periodicals as Topic/*standards
Publishing/standards
Republic of Korea
Research Design

Reference

1. Equator Network, Essential Resources for Writing and Publishing Health Research. Web site. Accessed July 3, 2016. http://www.equatornetwork.org/.

2. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD Initiative. Radiology. 2003; 226:24–28.

3. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015; 351:h5527.

4. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: An Updated List of Essential Items for Reporting Diagnostic Accuracy Studies. Radiology. 2015; 277:826–832.

5. Wikipedia. Open access journal. Web site. Accessed March 3, 2016. https://en.wikipedia.org/wiki/Open_access_journal.

6. Smith R. Peer review: reform or revolution? BMJ. 1997; 315:759–760.

7. Goldbeck-Wood S. Evidence on peer review-scientific quality control or smokescreen? BMJ. 1999; 318:44–45.

8. Van Raan AFJ. The use of bibliometric analysis in research performance assessment and monitoring of interdisciplinary scientific developments. Technikfolgenabschätzung. 2003; 1:20–29.

9. Smidt N, Rutjes AW, van der Windt DA, Ostelo RW, Reitsma JB, Bossuyt PM, et al. Quality of reporting of diagnostic accuracy studies. Radiology. 2005; 235:347–353.

10. Dilauro M, McInnes MD, Korevaar DA, van der Pol CB, Petrcich W, Walther S, et al. Is There an Association between STARD Statement Adherence and Citation Rate? Radiology. 2016; 280:62–67.

11. Wilczynski NL. Quality of reporting of diagnostic accuracy studies: no change since STARD statement publication--before-and-after study. Radiology. 2008; 248:817–823.

12. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011; 155:529–536.

13. Rutjes AW, Reitsma JB, Vandenbroucke JP, Glas AS, Bossuyt PM. Case-control and two-gate designs in diagnostic accuracy studies. Clin Chem. 2005; 51:1335–1341.

14. Knottnerus JA, Muris JW. Assessment of the accuracy of diagnostic tests: the cross-sectional study. J Clin Epidemiol. 2003; 56:1118–1128.

15. Bossuyt PM, Leeflang MM. Developing Criteria for Including Studies. In : Deeks JJ, Bossuyt PM, Gatsonis C, editors. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 0.4 [updated September 2008]. Oxford: The Cochrane Collaboration;2008.

16. Song YS, Choi SH, Park CK, Yi KS, Lee WJ, Yun TJ, et al. True progression versus pseudoprogression in the treatment of glioblastomas: a comparison study of normalized cerebral blood volume and apparent diffusion coefficient by histogram analysis. Korean J Radiol. 2013; 14:662–672.

17. Moon JH, Park EA, Lee W, Yin YH, Chung JW, Park JH, et al. The diagnostic accuracy, image quality and radiation dose of 64-slice dual-source CT in daily practice: a single institution's experience. Korean J Radiol. 2011; 12:308–318.

18. Lee KH, Lee JM, Park JH, Kim JH, Park HS, Yu MH, et al. MR imaging in patients with suspected liver metastases: value of liver-specific contrast agent gadoxetic acid. Korean J Radiol. 2013; 14:894–904.

19. Lee DH, Lee JM, Klotz E, Kim SJ, Kim KW, Han JK, et al. Detection of recurrent hepatocellular carcinoma in cirrhotic liver after transcatheter arterial chemoembolization: value of quantitative color mapping of the arterial enhancement fraction of the liver. Korean J Radiol. 2013; 14:51–60.

20. Ko ES, Han BK, Kim SM, Ko EY, Jang M, Lyou CY, et al. Comparison of new and established full-field digital mammography systems in diagnostic performance. Korean J Radiol. 2013; 14:164–170.

21. Kim SA, Chang JM, Cho N, Yi A, Moon WK. Characterization of breast lesions: comparison of digital breast tomosynthesis and ultrasonography. Korean J Radiol. 2015; 16:229–238.

22. Kim JI, Kim YH, Lee KH, Kim SY, Lee YJ, Park YS, et al. Type-specific diagnosis and evaluation of longitudinal tumor extent of borrmann type IV gastric cancer: CT versus gastroscopy. Korean J Radiol. 2013; 14:597–606.

23. Kim J, Kim YH, Lee KH, Lee YJ, Park JH. Diagnostic performance of CT angiography in patients visiting emergency department with overt gastrointestinal bleeding. Korean J Radiol. 2015; 16:541–549.

24. Jung SI, Park HS, Yim Y, Jeon HJ, Yu MH, Kim YJ, et al. Added value of using a CT coronal reformation to diagnose adnexal torsion. Korean J Radiol. 2015; 16:835–845.

25. Jung SI, Park HS, Kim YJ, Jeon HJ. Multidetector computed tomography for the assessment of adnexal mass: is unenhanced CT scan necessary? Korean J Radiol. 2014; 15:72–79.

26. Iannicelli E, Di Renzo S, Ferri M, Pilozzi E, Di Girolamo M, Sapori A, et al. Accuracy of high-resolution MRI with lumen distention in rectal cancer staging and circumferential margin involvement prediction. Korean J Radiol. 2014; 15:37–44.

27. Chung SY, Park SH, Lee SS, Lee JH, Kim AY, Park SK, et al. Comparison between CT colonography and double-contrast barium enema for colonic evaluation in patients with renal insufficiency. Korean J Radiol. 2012; 13:290–299.

28. Bang SH, Lee JY, Woo H, Joo I, Lee ES, Han JK, et al. Differentiating between adenomyomatosis and gallbladder cancer: revisiting a comparative study of high-resolution ultrasound, multidetector CT, and MR imaging. Korean J Radiol. 2014; 15:226–234.

29. Zhang JJ, Liu T, Feng Y, Wu WF, Mou CY, Zhai LH. Diagnostic value of 64-slice dual-source CT coronary angiography in patients with atrial fibrillation: comparison with invasive coronary angiography. Korean J Radiol. 2011; 12:416–423.

30. Yu H, Cui JL, Cui SJ, Sun YC, Cui FZ. Differentiating benign from malignant bone tumors using fluid-fluid level features on magnetic resonance imaging. Korean J Radiol. 2014; 15:757–763.

31. Yoon SH, Goo JM, Jung J, Hong H, Park EA, Lee CH, et al. Computer-aided classification of visual ventilation patterns in patients with chronic obstructive pulmonary disease at two-phase xenon-enhanced CT. Korean J Radiol. 2014; 15:386–396.

32. Yoon HJ, Chung MJ, Hwang HS, Moon JW, Lee KS. Adaptive statistical iterative reconstruction-applied ultra-low-dose CT with radiography-comparable radiation dose: usefulness for lung nodule detection. Korean J Radiol. 2015; 16:1132–1141.

33. Yoo JY, Chung MJ, Choi B, Jung HN, Koo JH, Bae YA, et al. Digital tomosynthesis for PNS evaluation: comparisons of patient exposure and image quality with plain radiography. Korean J Radiol. 2012; 13:136–143.

34. Yi J, Lee EH, Kwak JJ, Cha JG, Jung SH. Retrieval rate and accuracy of ultrasound-guided 14-G semi-automated core needle biopsy of breast microcalcifications. Korean J Radiol. 2014; 15:12–19.

35. Wu EH, Chen YL, Wu YM, Huang YT, Wong HF, Ng SH. CT-guided core needle biopsy of deep suprahyoid head and neck lesions. Korean J Radiol. 2013; 14:299–306.

36. Wu CH, Huang CC, Wang LJ, Wong YC, Wang CJ, Lo WC, et al. Value of CT in the discrimination of fatal from non-fatal stercoral colitis. Korean J Radiol. 2012; 13:283–289.

37. Woo S, Kim SY, Cho JY, Kim SH. Shear wave elastography for detection of prostate cancer: a preliminary study. Korean J Radiol. 2014; 15:346–355.

38. Ucar M, Guryildirim M, Tokgoz N, Kilic K, Borcek A, Oner Y, et al. Evaluation of aqueductal patency in patients with hydrocephalus: three-dimensional high-sampling-efficiency technique (SPACE) versus two-dimensional turbo spin echo at 3 Tesla. Korean J Radiol. 2014; 15:827–835.

39. Tong T, Zhenwei Y, Xiaoyuan F. Transient ischemic attack and stroke can be differentiated by analyzing the diffusion tensor imaging. Korean J Radiol. 2011; 12:280–288.

40. Tomozawa Y, Inaba Y, Yamaura H, Sato Y, Kato M, Kanamoto T, et al. Clinical value of CT-guided needle biopsy for retroperitoneal lesions. Korean J Radiol. 2011; 12:351–357.

41. Sohn CH, Lee HP, Park JB, Chang HW, Kim E, Kim E, et al. Imaging findings of brain death on 3-tesla MRI. Korean J Radiol. 2012; 13:541–549.

42. Shin JE, Choi HJ, Kim MH, Cho KS. The serum CA-125 concentration data assists in evaluating CT imaging information when used to differentiate borderline ovarian tumor from malignant epithelial ovarian tumors. Korean J Radiol. 2011; 12:456–462.

43. Ryoo I, Suh S, Lee YH, Seo HS, Seol HY. Comparison of ultrasonographic findings of biopsy-proven tuberculous lymphadenitis and Kikuchi disease. Korean J Radiol. 2015; 16:767–775.

44. Rief M, Stenzel F, Kranz A, Schlattmann P, Dewey M. Time efficiency and diagnostic accuracy of new automated myocardial perfusion analysis software in 320-row CT cardiac imaging. Korean J Radiol. 2013; 14:21–29.

45. Park SO, Seo JB, Kim N, Lee YK, Lee J, Kim DS. Comparison of usual interstitial pneumonia and nonspecific interstitial pneumonia: quantification of disease severity and discrimination between two diseases on HRCT using a texture-based automated system. Korean J Radiol. 2011; 12:297–307.

46. Park B, Kim HK, Choi YS, Kim J, Zo JI, Choi JY, et al. Prediction of pathologic grade and prognosis in mucoepidermoid carcinoma of the lung using ¹⁸F-FDG PET/CT. Korean J Radiol. 2015; 16:929–935.

47. Niu XK, Bhetuwal A, Yang HF. CT-guided core needle biopsy of pleural lesions: evaluating diagnostic yield and associated complications. Korean J Radiol. 2015; 16:206–212.

48. Lyou CY, Cho N, Kim SM, Jang M, Park JS, Baek SY, et al. Computer-aided evaluation of breast MRI for the residual tumor extent and response monitoring in breast cancer patients receiving neoadjuvant chemotherapy. Korean J Radiol. 2011; 12:34–43.

49. Lv P, Mahyoub R, Lin X, Chen K, Chai W, Xie J. Differentiating pancreatic ductal adenocarcinoma from pancreatic serous cystadenoma, mucinous cystadenoma, and a pseudocyst with detailed analysis of cystic features on CT scans: a preliminary study. Korean J Radiol. 2011; 12:187–195.

50. Luczynńska E, Heinze-Paluchowska S, Dyczek S, Blecharz P, Rys J, Reinfuss M. Contrast-enhanced spectral mammography: comparison with conventional mammography and histopathology in 152 women. Korean J Radiol. 2014; 15:689–696.

51. Lu Q, Huang BJ, Wang WP, Li CX, Xue LY. Qualitative and quantitative analysis with contrast-enhanced ultrasonography: diagnosis value in hypoechoic renal angiomyolipoma. Korean J Radiol. 2015; 16:334–341.

52. Liu X, Peng W, Zhou L, Wang H. Biexponential apparent diffusion coefficients values in the prostate: comparison among normal tissue, prostate cancer, benign prostatic hyperplasia and prostatitis. Korean J Radiol. 2013; 14:222–232.

53. Lim HJ, Chung MJ, Lee G, Yie M, Shin KE, Moon JW, et al. Interpretation of digital chest radiographs: comparison of light emitting diode versus cold cathode fluorescent lamp backlit monitors. Korean J Radiol. 2013; 14:968–976.

54. Lee YH, Kim DW, In HS, Park JS, Kim SH, Eom JW, et al. Differentiation between benign and malignant solid thyroid nodules using an US classification system. Korean J Radiol. 2011; 12:559–567.

55. Lee SJ, Lee WW, Kim SE. Bone positron emission tomography with or without CT is more accurate than bone scan for detection of bone metastasis. Korean J Radiol. 2013; 14:510–519.

56. Lee S, Lee YH, Chung TS, Jeong EK, Kim S, Yoo YH, et al. Accuracy of diffusion tensor imaging for diagnosing cervical spondylotic myelopathy in patients showing spinal cord compression. Korean J Radiol. 2015; 16:1303–1312.

57. Lee MH, Kim SH, Kim H, Lee MW, Lee WJ. Differentiating focal eosinophilic infiltration from metastasis in the liver with gadoxetic acid-enhanced magnetic resonance imaging. Korean J Radiol. 2011; 12:439–449.

58. Lee KH, Goo JM, Park CM, Lee HJ, Jin KN. Computer-aided detection of malignant lung nodules on chest radiographs: effect on observers' performance. Korean J Radiol. 2012; 13:564–571.

59. Lee JH, Yoon YC, Jee S, Kwon JW, Cha JG, Yoo JC. Comparison of three-dimensional isotropic and two-dimensional conventional indirect MR arthrography for the diagnosis of rotator cuff tears. Korean J Radiol. 2014; 15:771–780.

60. Lee JH, Byun JH, Kim JH, Lee SS, Kim HJ, Lee MG. Solid pancreatic tumors with unilocular cyst-like appearance on CT: differentiation from unilocular cystic tumors using CT. Korean J Radiol. 2014; 15:704–711.

61. Lee JE, Lee JM, Lee KB, Yoon JH, Shin CI, Han JK, et al. Noninvasive assessment of hepatic fibrosis in patients with chronic hepatitis B viral infection using magnetic resonance elastography. Korean J Radiol. 2014; 15:210–217.

62. Lee EK, Choi SH, Yun TJ, Kang KM, Kim TM, Lee SH, et al. Prediction of response to concurrent chemoradiotherapy with temozolomide in glioblastoma: application of immediate post-operative dynamic susceptibility contrast and diffusion-weighted MR imaging. Korean J Radiol. 2015; 16:1341–1348.

63. Koo JH, Kim CK, Choi D, Park BK, Kwon GY, Kim B. Diffusion-weighted magnetic resonance imaging for the evaluation of prostate cancer: optimal B value at 3T. Korean J Radiol. 2013; 14:61–69.

64. Kim YP, Kannengiesser S, Paek MY, Kim S, Chung TS, Yoo YH, et al. Differentiation between focal malignant marrow-replacing lesions and benign red marrow deposition of the spine with T2^*-corrected fat-signal fraction map using a three-echo volume interpolated breath-hold gradient echo Dixon sequence. Korean J Radiol. 2014; 15:781–791.

65. Kim SY, Moon SK, Jung DC, Hwang SI, Sung CK, Cho JY, et al. Pre-operative prediction of advanced prostatic cancer using clinical decision support systems: accuracy comparison between support vector machine and artificial neural network. Korean J Radiol. 2011; 12:588–594.

66. Kim SH, Kang BJ, Choi BG, Choi JJ, Lee JH, Song BJ, et al. Radiologists' performance for detecting lesions and the interobserver variability of automated whole breast ultrasound. Korean J Radiol. 2013; 14:154–163.

67. Kim MY, Cho N, Yi A, Koo HR, Yun BL, Moon WK. Sonoelastography in distinguishing benign from malignant complex breast mass and making the decision to biopsy. Korean J Radiol. 2013; 14:559–567.

68. Kim JE, Lee JY, Bae KS, Han JK, Choi BI. Acoustic radiation force impulse elastography for focal hepatic tumors: usefulness for differentiating hemangiomas from malignant tumors. Korean J Radiol. 2013; 14:743–753.

69. Kim HJ, Lee SH, Kang CH, Ryu JA, Shin MJ, Cho KJ, et al. Evaluation of the chondromalacia patella using a microscopy coil: comparison of the two-dimensional fast spin echo techniques and the three-dimensional fast field echo techniques. Korean J Radiol. 2011; 12:78–88.

70. Kim DW, Jung SL, Kim J, Ryu JH, Sung JY, Lim HK. Comparison between ultrasonography and computed tomography for detecting the pyramidal lobe of the thyroid gland: a prospective multicenter study. Korean J Radiol. 2015; 16:402–409.

71. Kang KM, Choi SI, Chun EJ, Kim JA, Youn TJ, Choi DJ. Coronary vasospastic angina: assessment by multidetector CT coronary angiography. Korean J Radiol. 2012; 13:27–33.

72. Kang KA, Kim YK, Kim E, Jeong WK, Choi D, Lee WJ, et al. T2-weighted liver MRI using the MultiVane technique at 3T: comparison with conventional T2-weighted MRI. Korean J Radiol. 2015; 16:1038–1046.

73. Jeh SK, Kim SH, Kang BJ. Comparison of the diagnostic performance of response evaluation criteria in solid tumor 1.0 with response evaluation criteria in solid tumor 1.1 on MRI in advanced breast cancer response evaluation to neoadjuvant chemotherapy. Korean J Radiol. 2013; 14:13–20.

74. Hur S, Kim HC, Chung JW, Kim MU, Kim JD, Kim GM, et al. Hepatocellular carcinomas smaller than 4 cm supplied by the intercostal artery: can we predict which intercostal artery supplies the tumor? Korean J Radiol. 2011; 12:693–699.

75. Chun KY, Choi YS, Lee SH, Kim JS, Young KW, Jeong MS, et al. Deltoid ligament and tibiofibular syndesmosis injury in chronic lateral ankle instability: magnetic resonance imaging evaluation at 3t and comparison with arthroscopy. Korean J Radiol. 2015; 16:1096–1103.

76. Choi HS, Kim AH, Ahn SS, Shin NY, Kim J, Lee SK. Glioma grading capability: comparisons among parameters from dynamic contrast-enhanced MRI and ADC value on DWI. Korean J Radiol. 2013; 14:487–492.

77. Cheung JY, Kim Y, Shim SS, Lim SM. Combined fluoroscopy- and CT-guided transthoracic needle biopsy using a C-arm cone-beam CT system: comparison with fluoroscopy-guided biopsy. Korean J Radiol. 2011; 12:89–96.

78. Ahn HS, Kim SM, Jang M, Yun BL, Kim B, Ko ES, et al. A new full-field digital mammography system with and without the use of an advanced post-processing algorithm: comparison of image quality and diagnostic performance. Korean J Radiol. 2014; 15:305–312.

79. Korevaar DA, Wang J, van Enst WA, Leeflang MM, Hooft L, Smidt N, et al. Reporting diagnostic accuracy studies: some improvements after 10 years of STARD. Radiology. 2015; 274:781–789.

80. Walther S, Schueler S, Tackmann R, Schuetz GM, Schlattmann P, Dewey M. Compliance with STARD checklist among studies of coronary CT angiography: systematic review. Radiology. 2014; 271:74–86.

81. Not-so-deep impact. Nature. 2005; 435:1003–1004.