J Korean Med Sci.  2021 Jul;36(28):e187. 10.3346/jkms.2021.36.e187.

Prediction of Neurological Outcomes in Out-of-hospital Cardiac Arrest Survivors Immediately after Return of Spontaneous Circulation: Ensemble Technique with Four Machine Learning Models

• Heo JH
• Kim T
• Shin J
• Suh GJ
• Kim J
• Jung YS
• Park SM
• Kim S
• For SNU CARE investigators

• KMID: 2518524
• DOI: http://doi.org/10.3346/jkms.2021.36.e187

Abstract

Background
We performed this study to establish a prediction model for 1-year neurological outcomes in out-of-hospital cardiac arrest (OHCA) patients who achieved return of spontaneous circulation (ROSC) immediately after ROSC using machine learning methods.
Methods
We performed a retrospective analysis of an OHCA survivor registry. Patients aged ≥ 18 years were included. Study participants who had registered between March 31, 2013 and December 31, 2018 were divided into a develop dataset (80% of total) and an internal validation dataset (20% of total), and those who had registered between January 1, 2019 and December 31, 2019 were assigned to an external validation dataset. Four machine learning methods, including random forest, support vector machine, ElasticNet and extreme gradient boost, were implemented to establish prediction models with the develop dataset, and the ensemble technique was used to build the final prediction model. The prediction performance of the model in the internal validation and the external validation dataset was described with accuracy, area under the receiver-operating characteristic curve, area under the precision-recall curve, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Futhermore, we established multivariable logistic regression models with the develop set and compared prediction performance with the ensemble models. The primary outcome was an unfavorable 1-year neurological outcome.
Results
A total of 1,207 patients were included in the study. Among them, 631, 139, and 153 were assigned to the develop, the internal validation and the external validation datasets, respectively. Prediction performance metrics for the ensemble prediction model in the internal validation dataset were as follows: accuracy, 0.9620 (95% confidence interval [CI],0.9352–0.9889); area under receiver-operator characteristics curve, 0.9800 (95% CI, 0.9612– 0.9988); area under precision-recall curve, 0.9950 (95% CI, 0.9860–1.0000); sensitivity, 0.9594 (95% CI, 0.9245–0.9943); specificity, 0.9714 (95% CI, 0.9162–1.0000); PPV, 0.9916 (95% CI, 0.9752–1.0000); NPV, 0.8718 (95% CI, 0.7669–0.9767). Prediction performance metrics for the model in the external validation dataset were as follows: accuracy, 0.8509 (95% CI, 0.7825–0.9192); area under receiver-operator characteristics curve, 0.9301 (95% CI, 0.8845–0.9756); area under precision-recall curve, 0.9476 (95% CI, 0.9087–0.9867); sensitivity, 0.9595 (95% CI, 0.9145–1.0000); specificity, 0.6500 (95% CI, 0.5022–0.7978); PPV, 0.8353 (95% CI, 0.7564–0.9142); NPV, 0.8966 (95% CI, 0.7857–1.0000). All the prediction metrics were higher in the ensemble models, except NPVs in both the internal and the external validation datasets.
Conclusion
We established an ensemble prediction model for prediction of unfavorable 1-year neurological outcomes in OHCA survivors using four machine learning methods. The prediction performance of the ensemble model was higher than the multivariable logistic regression model, while its performance was slightly decreased in the external validation dataset.

Keyword

Heart Arrest; Cardiopulmonary Resuscitation; Machine Learning

Figure

• 

  Fig. 1 Study flow chart.CPC = cerebral performance category.

• 

  Fig. 2 Receiver operating characteristic curves and precision-recall curves for the ensemble prediction model in various datasets. (A) and (B) in the original internal validation dataset, (C) and (D) in the external validation dataset, (E) and (F) in the cardiac subgroup internal validation dataset, (G) and (H) in the cardiac subgroup external validation dataset.

Reference

1. Holmberg MJ, Ross CE, Fitzmaurice GM, Chan PS, Duval-Arnould J, Grossestreuer AV, et al. Annual incidence of adult and pediatric in-hospital cardiac arrest in the United States. Circ Cardiovasc Qual Outcomes. 2019; 12(7):e005580. PMID: 31545574.

2. Hawkes C, Booth S, Ji C, Brace-McDonnell SJ, Whittington A, Mapstone J, et al. Epidemiology and outcomes from out-of-hospital cardiac arrests in England. Resuscitation. 2017; 110:133–140. PMID: 27865775.

3. Virani SS, Alonso A, Benjamin EJ, Bittencourt MS, Callaway CW, Carson AP, et al. Heart disease and stroke statistics-2020 update: a report from the American Heart Association. Circulation. 2020; 141(9):e139–e596. PMID: 31992061.

4. Nolan JP, Soar J, Cariou A, Cronberg T, Moulaert VR, Deakin CD, et al. European resuscitation council and European society of intensive care medicine guidelines for post-resuscitation care 2015: section 5 of the European resuscitation council guidelines for resuscitation 2015. Resuscitation. 2015; 95:202–222. PMID: 26477702.

5. Panchal AR, Bartos JA, Cabanas JG, Donnino MW, Drennan IR, Hirsch KG, et al. Part 3: adult basic and advanced life support: 2020 American Heart Association guidelines for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation. 2020; 142(16_):suppl_2. S366–S468. PMID: 33081529.

6. Kim YM, Park KN, Choi SP, Lee BK, Park K, Kim J, et al. Part 4. post-cardiac arrest care: 2015 Korean guidelines for cardiopulmonary resuscitation. Clin Exp Emerg Med. 2016; 3(Suppl):S27–S38. PMID: 27752644.

7. Dale CM, Sinuff T, Morrison LJ, Golan E, Scales DC. Understanding early decisions to withdraw life-sustaining therapy in cardiac arrest survivors. a qualitative investigation. Ann Am Thorac Soc. 2016; 13(7):1115–1122. PMID: 27073975.

8. Seki T, Tamura T, Suzuki M; SOS-KANTO 2012 Study Group. Outcome prediction of out-of-hospital cardiac arrest with presumed cardiac aetiology using an advanced machine learning technique. Resuscitation. 2019; 141:128–135. PMID: 31220514.

9. Kwon JM, Jeon KH, Kim HM, Kim MJ, Lim S, Kim KH, et al. Deep-learning-based out-of-hospital cardiac arrest prognostic system to predict clinical outcomes. Resuscitation. 2019; 139:84–91. PMID: 30978378.

10. Park JH, Shin SD, Song KJ, Hong KJ, Ro YS, Choi JW, et al. Prediction of good neurological recovery after out-of-hospital cardiac arrest: a machine learning analysis. Resuscitation. 2019; 142:127–135. PMID: 31362082.

11. Daou O, Winiszewski H, Besch G, Pili-Floury S, Belon F, Guillon B, et al. Initial pH and shockable rhythm are associated with favorable neurological outcome in cardiac arrest patients resuscitated with extracorporeal cardiopulmonary resuscitation. J Thorac Dis. 2020; 12(3):849–857. PMID: 32274152.

12. Kiehl EL, Amuthan R, Adams MP, Love TE, Enfield KB, Gimple LW, et al. Initial arterial pH as a predictor of neurologic outcome after out-of-hospital cardiac arrest: a propensity-adjusted analysis. Resuscitation. 2019; 139:76–83. PMID: 30946922.

13. Bender PR, Debehnke DJ, Swart GL, Hall KN. Serum potassium concentration as a predictor of resuscitation outcome in hypothermic cardiac arrest. Wilderness Environ Med. 1995; 6(3):273–282. PMID: 11990091.

14. Choi DS, Shin SD, Ro YS, Lee KW. Relationship between serum potassium level and survival outcome in out-of-hospital cardiac arrest using CAPTURES database of Korea: Does hypokalemia have good neurological outcomes in out-of-hospital cardiac arrest? Adv Clin Exp Med. 2020; 29(6):727–734. PMID: 32608580.

15. Tamura T, Suzuki M, Hayashida K, Sasaki J, Yonemoto N, Sakurai A, et al. Renal function and outcome of out-of-hospital cardiac arrest - multicenter prospective study (SOS-KANTO 2012 Study). Circ J. 2018; 83(1):139–146. PMID: 30333435.

16. D'Arrigo S, Cacciola S, Dennis M, Jung C, Kagawa E, Antonelli M, et al. Predictors of favourable outcome after in-hospital cardiac arrest treated with extracorporeal cardiopulmonary resuscitation: a systematic review and meta-analysis. Resuscitation. 2017; 121:62–70. PMID: 29020604.

17. Li DC, Hu SC, Lin LS, Yeh CW. Detecting representative data and generating synthetic samples to improve learning accuracy with imbalanced data sets. PLoS One. 2017; 12(8):e0181853. PMID: 28771522.

18. Callaway CW, Donnino MW, Fink EL, Geocadin RG, Golan E, Kern KB, et al. Part 8: post-cardiac arrest care: 2015 American Heart Association guidelines update for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation. 2015; 132(18):Suppl 2. S465–S482. PMID: 26472996.

19. Graf J, Mühlhoff C, Doig GS, Reinartz S, Bode K, Dujardin R, et al. Health care costs, long-term survival, and quality of life following intensive care unit admission after cardiac arrest. Crit Care. 2008; 12(4):R92. PMID: 18638367.

20. Efendijev I, Folger D, Raj R, Reinikainen M, Pekkarinen PT, Litonius E, et al. Outcomes and healthcare-associated costs one year after intensive care-treated cardiac arrest. Resuscitation. 2018; 131:128–134. PMID: 29958958.

21. Damluji AA, Al-Damluji MS, Pomenti S, Zhang TJ, Cohen MG, Mitrani RD, et al. Health care costs after cardiac arrest in the United States. Circ Arrhythm Electrophysiol. 2018; 11(4):e005689. PMID: 29654127.

22. Hong JY, Lee DH, Oh JH, Lee SH, Choi YH, Kim SH, et al. Grey-white matter ratio measured using early unenhanced brain computed tomography shows no correlation with neurological outcomes in patients undergoing targeted temperature management after cardiac arrest. Resuscitation. 2019; 140:161–169. PMID: 30953628.