J Educ Eval Health Prof.  2020;17:14. 10.3352/jeehp.2020.17.14.

Using the Angoff method to set a standard on mock exams for the Korean Nursing Licensing Examination

• Yim MK
• Shin S

• KMID: 2502199
• DOI: http://doi.org/10.3352/jeehp.2020.17.14

Abstract

Purpose
This study explored the possibility of using the Angoff method, in which panel experts determine the cut score of an exam, for the Korean Nursing Licensing Examination (KNLE). Two mock exams for the KNLE were analyzed. The Angoff standard setting procedure was conducted and the results were analyzed. We also aimed to examine the procedural validity of applying the Angoff method in this context.
Methods
For both mock exams, we set a pass-fail cut score using the Angoff method. The standard setting panel consisted of 16 nursing professors. After the Angoff procedure, the procedural validity of establishing the standard was evaluated by investigating the responses of the standard setters.
Results
The descriptions of the minimally competent person for the KNLE were presented at the levels of general and subject performance. The cut scores of first and second mock exams were 74.4 and 76.8, respectively. These were higher than the traditional cut score (60% of the total score of the KNLE). The panel survey showed very positive responses, with scores higher than 4 out of 5 points on a Likert scale.
Conclusion
The scores calculated for both mock tests were similar, and were much higher than the existing cut scores. In the second simulation, the standard deviation of the Angoff rating was lower than in the first simulation. According to the survey results, procedural validity was acceptable, as shown by a high level of confidence. The results show that determining cut scores by an expert panel is an applicable method.

Keyword

Angoff method; Nurses; Nursing licensure; Standard setting; Republic of Korea

Reference

References

1. Lee G. A psychometric approach to setting a passing score on Korean National Medical Licensing Examination. J Educ Eval Health Prof. 2004; 1:5–14. https://doi.org/10.3352/jeehp.2004.1.1.5.

2. National Council of State Boards of Nursing. Test plan for the National Council Licensure Examination for Registered Nurses [Internet]. Chicago (IL): National Council of State Boards of Nursing;2019. [2020 Apr 1]. Available from: http://www.ncsbn.org.

3. Seo YO, Park IS, Hwang SY, Kang SY, Shin SJ. A study of policy suggestions for nurturing and distributing excellent nurses. Seoul: Korea Health Personnel Licensing Examination;2017. Report No.: research report RE02-1706-05.

4. Jung SY, Lee YH, Seo YO, Han CO, Kim MK. A study on improvement of qualification system for graduates from foreign universities. Seoul: Korea Health Personnel Licensing Examination;2019. Report No.: research report RE12-1907-00.

5. Kwak CY, Yoon SY. A study on item management system of NCLEX-RN in USA. Seoul: Korea Health Personnel Licensing Examination;2010. Report No.: research report RE02-1004-05.

6. Kang SY, Kim KH, Kim KS, Park IS, Park HS, Seo YO, Ahn SY. A study on the minimum competency for nursing university graduates and application of national licensing examination. Seoul: Korea Health Personnel Licensing Examination;2015. Report No.: research report RE02-1607-05.

7. Cizek G, Bunch MB. Standard setting. TJ Seong . Seoul: Hakjisa;2011.

8. Angoff WH. Scales, norms and equivalent scores. In : Thorndike RL, editor. Educational measurement. 2nd ed. Washington (DC): American Council on Education;1971. p. 508–600.

9. Berk RA. A consumer’s guide to setting performance standards on criterion-referenced tests. Rev Educ Res. 1986; 56:137–172. https://doi.org/10.3102/00346543056001137.

10. Cizek GJ. Setting performance standards: concepts, methods, and perspectives. Mahwah (NJ): Lawrence Erlbaum Associate;2001.

11. Korea Health Personnel Licensing Examination. Item analysis of 58th National Licensing Examination for Nurse in 2018 [Internet]. Seoul: Korea Health Personnel Licensing Examination;2018. [cited 2020 Apr 1]. Available from:http://www.kuksiwon.or.kr.

12. Kim MA, Lee ES, Yu SY. 7-Day National Licensing Examination for Nurse. Seoul: EDUPD;2018.

13. Yim M. Comparison of results between modified-Angoff and bookmark methods for estimating cut score of the Korean medical licensing examination. Korean J Med Educ. 2018; 30:347–357. https://doi.org/10.3946/kjme.2018.110.

14. Ahn DS, Park JH, Yim MK, Lee JH. A study about standard setting for national licensing exams for health professions. Seoul: Korea Health Personnel Licensing Examination;2018. Report No.: research report RE12-1815-00.

15. Park J, Ahn DS, Yim MK, Lee J. Comparison of standard-setting methods for the Korea Radiological technologist Licensing Examination: Angoff, Ebel, bookmark, and Hofstee. J Educ Eval Health Prof. 2018; 15:32. https://doi.org/10.3352/jeehp.2018.15.32.

16. Ferdous AA, Plake BS. The use of subsets of test questions in an Angoff standard-setting method. Educ Psycholl Meas. 2005; 65:185–201. https://doi.org/10.1177/0013164404264852.

17. Buckendahl CW, Ferdous AA, Gerrow J. Recommending cut scores with a subset of items: an empirical illustration. Pract Assess Res Eval. 2010; 15:6. https://doi.org/10.7275/tv3s-cz67.