Korean J Urol.  2013 May;54(5):289-296. 10.4111/kju.2013.54.5.289.

Assessing the Quality of Randomized Controlled Urological Trials Conducted by Korean Medical Institutions

• Chung JH
• Lee SW

• KMID: 2061509
• DOI: http://doi.org/10.4111/kju.2013.54.5.289

Abstract

PURPOSE
To assess the quality of randomized controlled urological trials conducted by Korean medical institutions.
MATERIALS AND METHODS
Quality assessment was conducted by using the Jadad scale; in addition, the van Tulder scale and the Cochrane Collaboration risk of bias tool were used as individual indices. All assessments were performed by two reviewers. If the outcomes differed, the two reviewers and a third reviewer adjusted the discrepancy in the results through discussion. Starting from 1986, a quality analysis of randomized controlled trials (RCTs) was conducted in 1-year and 5-year units. The quality assessment was conducted by subject, type of intervention, presence of double blinding, presence of funding, and review by an Institutional Review Board (IRB).
RESULTS
Whereas the number of RCTs published has gradually increased, there was no significant difference in the quality of the RCTs according to publication year. Drug studies, double-blind studies, studies with funding, and studies reviewed by IRBs had higher quality scores and a higher percentage of high-quality RCTs than did other studies. Thirty-six RCTs were published in journals included in the Science Citation Index and 20 RCTs were published in journals included in the Science Citation Index Expanded. The largest number of RCTs (32.32%) were published by the Korean Journal of Urology.
CONCLUSIONS
A quantitative increase was observed in RCTs over time, but no qualitative improvement in the RCTs was observed. It seems necessary to put effort into the quality improvement of RCTs at the design stage.

Keyword

Journal article; Randomized controlled trial

MeSH Terms

Bias (Epidemiology)
Cooperative Behavior
Double-Blind Method
Ethics Committees, Research
Financial Management
Publications
Quality Improvement

Figure

• 

  FIG. 1 Flow sheet.

• 

  FIG. 2 Quality assessment of randomized controlled trials according to publication year by use of 1-year units. One-way analysis of variance. a:Chi-square test.

• 

  FIG. 3 Journals in which articles were published. SCI, science citation idex; SCIE, Science Citation Index Expanded.

• 

  FIG. 4 Medical institution of the corresponding author.

Reference

1. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. 1996. 312:71–72.

2. Uetani K, Nakayama T, Ikai H, Yonemoto N, Moher D. Quality of reports on randomized controlled trials conducted in Japan: evaluation of adherence to the CONSORT statement. Intern Med. 2009. 48:307–313.

3. Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA. 1996. 276:637–639.

4. Jackson JL, Srinivasan M, Rea J, Fletcher KE, Kravitz RL. The validity of peer review in a general medicine journal. PLoS One. 2011. 6:e22475.

5. Chung W, Lee KW, Hwang IH, Lee DH, Kim SY. Quality assessment of randomized controlled trials in the Journal of the Korean Academy of Family Medicine. Korean J Fam Med. 2009. 30:626–631.

6. Liberati A, Himel HN, Chalmers TC. A quality assessment of randomized control trials of primary treatment of breast cancer. J Clin Oncol. 1986. 4:942–951.

7. Autorino R, Borges C, White MA, Altunrende F, Perdona S, Haber GP, et al. Randomized clinical trials presented at the World Congress of Endourology: how is the quality of reporting? J Endourol. 2010. 24:2067–2073.

8. Lim SM, Shin ES, Lee SH, Seo KH, Jung YM, Jang JE. Tools for assessing quality and risk of bias by levels of evidence. J Korean Med Assoc. 2011. 54:419–429.

9. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials. 1996. 17:1–12.

10. Moher D, Cook DJ, Jadad AR, Tugwell P, Moher M, Jones A, et al. Assessing the quality of reports of randomised trials: implications for the conduct of meta-analyses. Health Technol Assess. 1999. 3:i–iv. 1–98.

11. Lee JY, Chung JH, Kang DH, Lee JW, Moon HS, Yoo TK, et al. Quality assessment of randomized controlled trials published in the Korean Journal of Urology over the past 20 years. Korean J Urol. 2011. 52:642–646.

12. van Tulder M, Furlan A, Bombardier C, Bouter L. Editorial Board of the Cochrane Collaboration Back Review Group. Updated method guidelines for systematic reviews in the cochrane collaboration back review group. Spine (Phila Pa 1976). 2003. 28:1290–1299.

13. Higgins JP, Green S, editors. Cochrane handbook for systematic reviews of interventions. Ver. 5.1.0 [Internet]. 2011. cited 2013 Jan 7. The Cochrane Collaboration;Available from: http://www.cochrane-handbook.org.

14. Moher D, Schulz KF, Altman D. CONSORT Group (Consolidated Standards of Reporting Trials). The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA. 2001. 285:1987–1991.

15. Armijo-Olivo S, Stiles CR, Hagen NA, Biondo PD, Cummings GG. Assessment of study quality for systematic reviews: a comparison of the Cochrane Collaboration Risk of Bias Tool and the Effective Public Health Practice Project Quality Assessment Tool: methodological research. J Eval Clin Pract. 2012. 18:12–18.

16. Kim SW, Choi YS, Ahn HS, Lee HY, Ahn DS, Lee YM. Quantity and quality assessment of randomized controlled trials published in five Korean medical journals, from 1980 to 2000. J Korean Acad Fam Med. 2004. 25:118–125.

17. Keech AC, Pike R, Granger RE, Gebski VJ. Interpreting the results of a clinical trial. Med J Aust. 2007. 186:318–319.

18. Hewitt C, Hahn S, Torgerson DJ, Watson J, Bland JM. Adequacy and reporting of allocation concealment: review of recent trials published in four general medical journals. BMJ. 2005. 330:1057–1058.

19. Schulz KF, Grimes DA. Allocation concealment in randomised trials: defending against deciphering. Lancet. 2002. 359:614–618.

20. Clifford TJ, Barrowman NJ, Moher D. Funding source, trial outcome and reporting quality: are they related? Results of a pilot study. BMC Health Serv Res. 2002. 2:18.