J Korean Med Sci.  2013 Mar;28(3):357-365. 10.3346/jkms.2013.28.3.357.

Assessment of the Quality of Clinical Practice Guidelines in Korea Using the AGREE Instrument

• Jo MW
• Lee JY
• Kim NS
• Kim SY
• Sheen S
• Kim SH
• Lee SI

• KMID: 1786931
• DOI: http://doi.org/10.3346/jkms.2013.28.3.357

Abstract

The objective of this study was to conduct the systematic evaluation of methodological quality of clinical practice guidelines (CPGs) in Korea. The authors conducted a very comprehensive literature search to identify potential CPGs for evaluation. CPGs were selected which were consistent with a predetermined criteria. Four reviewers evaluated the quality of the CPGs using the Appraisal of Guidelines, Research and Evaluation (AGREE) Instrument. AGREE item scores and standardized domain scores were calculated. The inter-rater reliability of each domain was evaluated using the intra-class correlation coefficient (ICC). Consequently, 66 CPGs were selected and their quality evaluated. ICCs for CPG appraisal using the AGREE Instrument ranged from 0.626 to 0.877. Except for the "Scope and Purpose" and "Clarity and Presentation domains", 80% of CPGs scored less than 40 in all other domains. This review shows that many Korean research groups and academic societies have made considerable efforts to develop CPGs, and the number of CPGs has increased over time. However, the quality of CPGs in Korea were not good according to the AGREE Instrument evaluation. Therefore, we should make more of an effort to ensure the high quality of CPGs.

Keyword

Clinical Practice Guideline; Quality Improvement

MeSH Terms

Databases, Factual
Humans
Practice Guidelines as Topic/*standards
Quality Control
Republic of Korea

Figure

• 

  Fig. 1 Flowchart showing selection of clinical practice guidelines.

• 

  Fig. 2 Distribution of standardized domain scores for 66 clinical practice guidelines. The top and bottom of the box represents the 75th (Q3) and 25th percentile (Q1), respectively, and the band near the middle of the box indicates the 50th percentile (the median). The upper and lower ends of the whisker represent Q3 + 1.5 × (interquartile range), and Q1-1.5 × (interquartile range), respectively. Small dot (○) represents outlier values which lies more than 1.5 times to 3.0 times interquartile range from either end of the whisker. Asterisk (*) represents extreme outlier values which lies more than 3 times interquartile range from either end of the whisker.

Reference

1. Field MJ, Lohr KN. Clinical practice guidelines: directions for a new program. 1990. Washington, D.C.: National Academy Press.

2. Pauly MV, Eisenberg JM, Radany MH, Erder MH, Feldman R, Schwartz JS. Paying physicians: options for controlling cost, volume, and intensity of services. 1992. Ann Arbor: Health Administration Press.

3. Burgers JS, Grol R, Klazinga NS, Mäkelä M, Zaat J. AGREE Collaboration. Towards evidence-based clinical practice: an international survey of 18 clinical guideline programs. Int J Qual Health Care. 2003. 15:31–45.

4. AGREE Collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003. 12:18–23.

5. Nagy E, Watine J, Bunting PS, Onody R, Oosterhuis WP, Rogic D, Sandberg S, Boda K, Horvath AR. IFCC Task Force on the Global Campaign for Diabetes Mellitus. Do guidelines for the diagnosis and monitoring of diabetes mellitus fulfill the criteria of evidence-based guideline development? Clin Chem. 2008. 54:1872–1882.

6. Zhang W, Moskowitz RW, Nuki G, Abramson S, Altman RD, Arden N, Bierma-Zeinstra S, Brandt KD, Croft P, Doherty M, et al. OARSI recommendations for the management of hip and knee osteoarthritis, part I: critical appraisal of existing treatment guidelines and systematic review of current research evidence. Osteoarthritis Cartilage. 2007. 15:981–1000.

7. Wijkstra J, Schubart CD, Nolen WA. Treatment of unipolar psychotic depression: the use of evidence in practice guidelines. World J Biol Psychiatry. 2009. 10:409–415.

8. Hurdowar A, Graham ID, Bayley M, Harrison M, Wood-Dauphinee S, Bhogal S. Quality of stroke rehabilitation clinical practice guidelines. J Eval Clin Pract. 2007. 13:657–664.

9. Van der Wees PJ, Hendriks EJ, Custers JW, Burgers JS, Dekker J, de Bie RA. Comparison of international guideline programs to evaluate and update the Dutch program for clinical guideline development in physical therapy. BMC Health Serv Res. 2007. 7:191.

10. Esandi ME, Ortiz Z, Chapman E, Dieguez MG, Mejía R, Bernztein R. Production and quality of clinical practice guidelines in Argentina (1994-2004): a cross-sectional study. Implement Sci. 2008. 3:43.

11. Cho IH, Yoo HK, Son JW, Yoo HJ, Koo YJ, Chung US, Ahn DH, Ahn JS. The Korean practice parameter for the treatment of pervasive developmental disorders: pharmacological treatment. J Korean Acad Child Adolesc Psychiatry. 2007. 18:109–116.

12. Lee KS, Kim DJ. Guideline Committee of the Korean Association for the Study of the Liver. Management for chronic hepatitis B. Korean J Hepatol. 2007. 13:447–488.

13. The Korean Society of Nephrology. Clinical practice guideline for chronic renal disease. Korean J Nephrol. 2009. 28:S1–S98.

14. Choi CH, Jung SA, Lee BI, Lee KM, Kim JS, Han DS. IBD Study Group of the Korean Association of the Study of Intestinal Diseases. Diagnostic guideline of ulcerative colitis. Korean J Gastroenterol. 2009. 53:145–160.

15. Fervers B, Burgers JS, Haugh MC, Brouwers M, Browman G, Cluzeau F, Philip T. Predictors of high quality clinical practice guidelines: examples in oncology. Int J Qual Health Care. 2005. 17:123–132.

16. Alonso-Coello P, Irfan A, Solà I, Gich I, Delgado-Noguera M, Rigau D, Tort S, Bonfill X, Burgers J, Schunemann H. The quality of clinical practice guidelines over the last two decades: a systematic review of guideline appraisal studies. Qual Saf Health Care. 2010. 19:e58.

17. MacDermid JC, Brooks D, Solway S, Switzer-McIntyre S, Brosseau L, Graham ID. Reliability and validity of the AGREE instrument used by physical therapists in assessment of clinical practice guidelines. BMC Health Serv Res. 2005. 5:18.

18. Shimbo T, Fukui T, Ishioka C, Okamoto K, Okamoto T, Kameoka S, Sato A, Toi M, Matsui K, Mayumi T, et al. Quality of guideline development assessed by the Evaluation Committee of the Japan Society of Clinical Oncology. Int J Clin Oncol. 2010. 15:227–233.

19. Nast A, Spuls PH, Ormerod AD, Reytan N, Saiag PH, Smith CH, Rzany B. A critical appraisal of evidence-based guidelines for the treatment of psoriasis vulgaris: 'AGREE-ing' on a common base for European evidence-based psoriasis treatment guidelines. J Eur Acad Dermatol Venereol. 2009. 23:782–787.

20. Wennekes L, Hermens RP, van Heumen K, Runde V, Schoelen H, Wollersheim HC, Grol RP, de Mulder PH, Ottevanger PB. Possibilities for transborder cooperation in breast cancer care in Europe: a comparative analysis regarding the content, quality and evidence use of breast cancer guidelines. Breast. 2008. 17:464–471.

21. Stone MA, Wilkinson JC, Charpentier G, Clochard N, Grassi G, Lindblad U, Müller UA, Nolan J, Rutten GE, Khunti K. Evaluation and comparison of guidelines for the management of people with type 2 diabetes from eight European countries. Diabetes Res Clin Pract. 2010. 87:252–260.

22. Kinnunen-Amoroso M, Pasternack I, Mattila S, Parantainen A. Evaluation of the practice guidelines of Finnish Institute of Occupational Health with AGREE instrument. Ind Health. 2009. 47:689–693.

23. Tan JK, Wolfe BJ, Bulatovic R, Jones EB, Lo AY. Critical appraisal of quality of clinical practice guidelines for treatment of psoriasis vulgaris, 2006-2009. J Invest Dermatol. 2010. 130:2389–2395.

24. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, Fervers B, Graham ID, Grimshaw J, Hanna SE, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ. 2010. 182:E839–E842.

25. Ferket BS, Genders TS, Colkesen EB, Visser JJ, Spronk S, Steyerberg EW, Hunink MG. Systematic review of guidelines on imaging of asymptomatic coronary artery disease. J Am Coll Cardiol. 2011. 57:1591–1600.