J Korean Soc Emerg Med.  2019 Feb;30(1):100-110. 10.0000/jksem.2019.30.1.100.

The learner-adjusted assessment tool for endotracheal intubation considering examiners' different expectations of competence

• Kang HY
• Han KS
• Lee SW
• Choi HJ
• Lim TH
• Kim CW
• Chang CH
• Kim SJ

• KMID: 2441131
• DOI: http://doi.org/10.0000/jksem.2019.30.1.100

Abstract

OBJECTIVE
Endotracheal intubation (ETI) is the most reliable way to manage the airway. Stepwise deliberate practice and mastery training is essential in maintaining and promoting the skill of intubation. This study was conducted to identify differences in examiners' expectations regarding competent skill performance and to develop learner-adjusted assessment tools with appropriate levels according to student and resident learners based on the expectations and limited observation of performance by examiners.
METHODS
This was an exploratory, psychometric study using a simple airway part task trainer. The draft ETI assessment tool from the literatures, previous tools, and the preliminary learner-adjusted assessment tool for students and residents were developed and analyzed. Knowledge-based and competence-based items for each learner group were identified based on experts' expectations. The final learner-adjusted tools were refined through analyzing the content validity, internal consistency, and interrater reliability based on assessing the observed performance of 14 students and 12 residents by ten experts.
RESULTS
The preliminary student-adjusted assessment tool and resident-adjusted assessment tool had 12 items on the checklist with a ternary scoring system and a ternary scoring checklist including 15 items, and an overall Global Rating Scale. The final student-adjusted assessment tool was composed of a ternary scoring checklist including 9 items (total CVI, 86.6%; Cronbach's α, 0.83; interrater reliability, 0.64). The resident-adjusted assessment, on the other hand, was also composed of a ternary scoring checklist including 12 items (total CVI, 86.4%; Cronbach's α, 0.7; interrater reliability, 0.78), in addition to global rating scale including "˜rating of the overall process'.
CONCLUSION
The experts had different expectations regarding the level of competence in each step according to learner groups with different levels of difficulty. Understanding the factors influencing assessments can provide a guide for teaching and objectively assessing to the examiner.

Keyword

Assessment; Validity; Reliability; Endotracheal intubation; Training; Psychometrics

MeSH Terms

Checklist
Hand
Humans
Intubation
Intubation, Intratracheal*
Mental Competency*
Psychometrics