Womens Health Nurs.  2024 Mar;30(1):18-25. 10.4069/whn.2024.03.12.

Implementing alternative estimation methods to evaluate the reliability of Likert-scale instruments

• Park CG

• KMID: 2554187
• DOI: http://doi.org/10.4069/whn.2024.03.12

Figure

• 

  Figure 1. Estimation of ordinal alpha using Jamovi software and an online program.

Reference

References

1. Kiwanuka F, Kopra J, Sak-Dankosky N, Nanyonga RC, Kvist T. Polychoric correlation with ordinal data in nursing research. Nurs Res. 2022; 71(6):469–476. https://doi.org/10.1097/NNR.0000000000000614.

2. Zumbo BD, Gadermann AM, Zeisser C. Ordinal versions of coefficients alpha and theta for Likert rating scales. J Mod Appl Stat Methods. 2007; 6(1):21–29. https://doi.org/10.56801/10.56801/v6.i.279.

3. Gadermann AM, Guhn M, Zumbo BD. Estimating ordinal reliability for Likert-type and ordinal item response data: a conceptual, empirical, and practical guide. Pract Assess Res Eval. 2012; 17:1–13. https://doi.org/10.7275/n560-j767.

4. Peters GJ. The Alpha and the omega of scale reliability and validity: why and how to abandon Cronbach’s alpha and the route towards more comprehensive assessment of scale quality. Eur Health Psychol. 2014; 16(2):56–69. https://doi.org/10.31234/osf.io/h47fv.

5. Kelley K, Pornprasertmanit S. Confidence intervals for population reliability coefficients: Evaluation of methods, recommendations, and software for composite measures. Psychol Methods. 2016; 21(1):69–92. https://doi.org/10.1037/a0040086.

6. Viladrich C, Angulo-Brunet A, Doval E. A journey around alpha and omega to estimate internal consistency reliability. Ann Psychol. 2017; 33(3):755–782. https://doi.org/10.6018/analesps.33.3.268401.

7. McNeish D. Thanks coefficient alpha, we'll take it from here. Psychol Methods. 2018; 23(3):412–433. https://doi.org/10.1037/met0000144.

8. Flora DB. Your coefficient alpha is probably wrong, but which coefficient omega is right? A tutorial on using R to obtain better reliability estimates. Adv Methods Pract Psychol Sci. 2020; 3(4):484–501. https://doi.org/10.1177/2515245920951747.

9. Doval E, Viladrich C, Angulo-Brunet A. Coefficient alpha: the resistance of a classic. Psicothema. 2023; 35(1):5–20. https://doi.org/10.7334/psicothema2022.321.

10. Green SB, Yang Y. Reliability of summed item scores using structural equation modeling: an alternative to coefficient alpha. Psychometrika. 2009; 74(1):155–167. https://doi.org/10.1007/s11336-008-9099-3.

11. Yang Y, Green SB. Evaluation of structural equation modeling estimates of reliability for scales with ordered categorical items. Methodology: Eur J Res Methods Behav Soc Sci. 2015; 11(1):23–34. https://doi.org/10.1027/1614-2241/a000087.

12. Revelle W, Condon DM. Reliability from α to ω: a tutorial. Psychol Assess. 2019; 31(12):1395–1411. https://doi.org/10.1037/pas0000754.

13. Chalmers RP. On misconceptions and the limited usefulness of ordinal alpha. Educ Psychol Meas. 2018; 78(6):1056–1071. https://doi.org/10.1177/0013164417727036.

14. Metsämuuronen J. Attenuation-corrected estimators of reliability. Appl Psychol Meas. 2022; 46(8):720–737. https://doi.org/10.1177/01466216221108131.

15. Metsämuuronen J. Typology of deflation-corrected estimators of reliability. Front Psychol. 2022; 13:891959. https://doi.org/10.3389/fpsyg.2022.891959.

16. Metsämuuronen J. Seeking the real reliability: why the traditional estimators of reliability usually fail in achievement testing and why the deflation-corrected coefficients could be better options. Pract Assess Res Eval. 2023; 28:Article 10. https://doi.org/10.7275/pare.1264.

17. Foster RC. KR20 and KR21 for some nondichotomous data (it’s not just Cronbach’s alpha). Educ Psychol Meas. 2021; 81(6):1172–1202. https://doi.org/10.1177/0013164421992535.

18. Lee JJ, Clarke CL. Nursing students' attitudes towards information and communication technology: an exploratory and confirmatory factor analytic approach. J Adv Nurs. 2015; 71(5):1181–1193. https://doi.org/10.1111/jan.12611.

19. Rodríguez-Martín B, Martin-Martin R, Suhonen R. Individualized Care Scale-patient: a Spanish validation study. Nurs Ethics. 2019; 26(6):1791–1804. https://doi.org/10.1177/0969733018769351.

20. Säfström E, Nasstrom L, Liljeroos M, Nordgren L, Årestedt K, Jaarsma T, et al. Patient Continuity of Care Questionnaire in a cardiac sample: a confirmatory factor analysis. BMJ Open. 2020; 10(7):e037129. https://doi.org/10.1136/bmjopen-2020-037129.

21. Sterner A, Säfström E, Palmér L, Ramstrand N, Hagiwara MA. Development and initial validation of an instrument to measure novice nurses' perceived ability to provide care in acute situations - PCAS. BMC Nurs. 2020; 19:13. https://doi.org/10.1186/s12912-020-0406-3.

22. Agrazal-García J, Gordon-de Isaacs L, Tuñón R. Design and validation of a questionnaire to measure hantavirus preventive practices in an endemic community. Rev Peru Med Exp Salud Publica. 2022; 39(1):47–54. https://doi.org/10.17843/rpmesp.2022.391.9740.

23. Schulc E, Gothe RM, Them C, Tufan İ, Mueller G. Self-reported questionnaire of functional health of home-dwelling elderly people living in Austria: based on the patient questionnaire of the standardized assessment of elderly people in primary care. J Nurs Meas. 2017; 25(2):238–256. https://doi.org/10.1891/1061-3749.25.2.238.

24. Barbaranelli C, Lee CS, Vellone E, Riegel B. Dimensionality and reliability of the self-care of heart failure index scales: further evidence from confirmatory factor analysis. Res Nurs Health. 2014; 37(6):524–537. https://doi.org/10.1002/nur.21623.

25. Juliá-Sanchis R, Cabañero-Martínez MJ, Leal-Costa C, Fernández-Alcántara M, Escribano S. Psychometric properties of the health professionals communication skills scale in university students of health sciences. Int J Environ Res Public Health. 2020; 17(20):7565. https://doi.org/10.3390/ijerph17207565.

26. Esteban-Burgos AA, Fernández-Alcántara M, Escribano S, Perpiñá-Galvañ J, Campos-Calderón CP, Cabañero-Martínez MJ. Psychometric properties of the Spanish version of the Fatigue Assessment Scale in caregivers of palliative care patients. J Clin Med. 2022; 11(14):3999. https://doi.org/10.3390/jcm11143999.

27. Maziero VG, Campos JA, Dias BM. Authentic Leadership Questionnaire applied to Brazilian nurses: evidence of validity. Rev Lat Am Enfermagem. 2022; 30:e3607. https://doi.org/10.1590/1518-8345.5868.3607.

28. Lucasjfriesen, Jonathon-love. Psychometrics & Post-Data Analysis for Jamovi [Internet]. GitHub, Inc.; 2023 [cited 2024 Mar 5]. Available from: https://github.com/lucasjfriesen/jamoviPsychoPDA.

29. Jorgensen TD, Pornprasertmanit S, Schoemann AM, Rosseel Y. semTools: Useful tools for structural equation modeling (R package Version 0.5-3) [Computer software]. Authors; 2022 [cited 2024 Mar 5]. Available from: https://CRAN.R-project.org/package=semTools.

30. Andersson B. Irtreliability: item response theory reliability. Version 0.1-1 [Computer software]. Author; 2022 [cited 2024 Mar 5]. Available from: https://CRAN.R-project.org/package=irtreliability.