Ann Surg Treat Res.  2024 Dec;107(6):315-326. 10.4174/astr.2024.107.6.315.

Validation of the Vascular Study Group of New England (VSGNE) risk prediction model for abdominal aortic aneurysm repair in Korea: a single-center retrospective study

• Kim HS
• Kim DH
• Kim DI
• Park JK
• Yang SS
• Park YJ

• KMID: 2562307
• DOI: http://doi.org/10.4174/astr.2024.107.6.315

Abstract

Purpose
The Vascular Study Group of New England (VSGNE) risk prediction model is a simple method for estimating risk for elective abdominal aortic aneurysm (AAA) repair. The model considers both treatment methods and the physical characteristics of the aneurysm type as well as comorbidities. This research aimed to validate its effectiveness by analyzing retrospective data on Korean patients.
Methods
Our single-center retrospective analysis included 1,227 patients who underwent elective open repair surgery (ORS) or endovascular aortic repair (EVAR) from 2005 to 2021. We assessed the discrimination of the risk score and the effects of several risk factors.
Results
Most patients (66.7%) were classified as low risk in the model, with only 5.6% considered high risk. The mean risk score was 2.81, significantly lower than reported in previous studies. The actual 30-day mortality was only 0.7%, less than the predicted 1.1%. The accuracy of the model in predicting 30-day mortality was statistically significant (area under the curve, 0.822). Patients with high scores were associated with significantly increased mortality (odds ratio, 3.9; P < 0.001). Factors such as advanced age, cerebrovascular disease, and elevated creatinine levels were influential in mortality outcomes. However, a significant difference was not found in short-term mortality between ORS and EVAR.
Conclusion
Although the VSGNE model is an objective tool for assessing death risk in elective AAA repair, the actual risk scores in our patient population were lower than predicted. To create a more representative tool for the Korean population, we suggest developing a novel model based on multicenter data collection.

Keyword

Abdominal aortic aneurysms; Calibration; In hospital mortality; Risk score; ROC curve; Validation

Figure

• 

  Fig. 1 (A) Score distributions of Samsung Medical Center (SMC, red line) and Vascular Study Group of New England (VSGNE) sample cohorts (gray line). The score distribution of the cohort sample used by Eslami et al. [6] in designing their risk prediction model and predictive distribution of the all-cohort population (dotted line). Distribution in the Vascular Quality Initiative (VQI) database for estimated external validation by Eslami et al. [5] (blue line). (B) Risk groups divided into low risk (1–3 points), medium risk (4–6 points), and high risk (≥7 points) based on score distribution and mortality cutoff (the cutoff value is shown in Fig. 3).

• 

  Fig. 2 Comparison of observed and predicted mortality based on the Vascular Study Group of New England risk predictive model. (A) Observed 30-day mortality vs. predicted mortality of patients in the endovascular aortic repair (EVAR) and open repair surgery (ORS) groups. (B) Observed 30-day mortality vs. predicted mortality based on the American Society of Anesthesiologists (ASA) physical status classification.

• 

  Fig. 3 The receiver operating characteristics curve (ROC) for the area under the curve (AUC) values for the Vascular Study Group of New England risk predictive score regarding short-term mortality. (A) The 30-day mortality ROC of all patients. (B) The 1-year mortality ROC of all patients. (C) The 1-year mortality ROC of the EVAR and ORS groups. (D) Calculated Delong-adjusted AUC for each curve and Youden’s cutoff value.

• 

  Fig. 4 Performance of the Vascular Study Group of New England risk prediction model (quantiles). The plot comparing actual and predicted mortality after stratifying patients based on score. From the 1st to the 4th quantiles, the difference in 30-day mortality was minimal and 1-year mortality tended to be underestimated. The 5th quantile (scores > 9 points) showed a tendency to overestimate both 30-day and 1-year mortality scores.

• 

  Fig. 5 Logistic regression and Cox regression analyses. (A) Odds ratio (OR) and hazard ratio (HR) plot of multiple analyses (1-year mortality). (B) Subgroup multiple analysis for 1-year mortality. (C) Univariable analysis for Vascular Study Group of New England risk score groups: low (1–3 points), medium (4–6 points), and high (≥7 points). AAA, abdominal aortic aneurysm; BMI, body mass index; CVD, cardiovascular disease; COPD, chronic obstructive pulmonary disease; EVAR, endovascular aortic repair; ORS, open repair surgery; CI, confidence interval.

• 

  Fig. 6 Kaplan-Meier analysis. Survival curve based on stratification with Vascular Study Group of New England risk score groups: Risk group 3: low (1–3 points), group 2: medium (4–6 points), and group 1: high (≥7 points). (A) The 30-day survival curve for all patients. (B) The 1-year survival curve for all patients. (C) The 1-year survival curve for the endovascular aortic repair (EVAR) group. There was no high-risk group in EVAR. (D) The 1-year survival curve for the open repair surgery (ORS) group. (E) Mortality rate(%) based on stratified risk score.

Reference

1. Chaikof EL, Dalman RL, Eskandari MK, Jackson BM, Lee WA, Mansour MA, et al. The Society for Vascular Surgery practice guidelines on the care of patients with an abdominal aortic aneurysm. J Vasc Surg. 2018; 67:2–77. PMID: 29268916.

2. AlShakarchi J, Fairhead J, Rajagopalan S, Pherwani A, Jaipersad A. Impact of frailty on outcomes in patients undergoing open abdominal aortic aneurysm repair. Ann Vasc Surg. 2020; 67:100–104. PMID: 31743784.

3. Grant SW, Grayson AD, Mitchell DC, McCollum CN. Evaluation of five risk prediction models for elective abdominal aortic aneurysm repair using the UK National Vascular Database. Br J Surg. 2012; 99:673–679. PMID: 22415901.

4. Eslami MH, Rybin DV, Doros G, Siracuse JJ, Farber A. External validation of Vascular Study Group of New England risk predictive model of mortality after elective abdominal aorta aneurysm repair in the Vascular Quality Initiative and comparison against established models. J Vasc Surg. 2018; 67:143–150. PMID: 28807384.

5. Cronenwett JL, Likosky DS, Russell MT, Eldrup-Jorgensen J, Stanley AC, Nolan BW, et al. A regional registry for quality assurance and improvement: the Vascular Study Group of Northern New England (VSGNNE). J Vasc Surg. 2007; 46:1093–1102. PMID: 17950568.

6. Eslami MH, Rybin D, Doros G, Kalish JA, Farber A. Vascular Study Group of New England. Comparison of a Vascular Study Group of New England risk prediction model with established risk prediction models of in-hospital mortality after elective abdominal aortic aneurysm repair. J Vasc Surg. 2015; 62:1125–1133. PMID: 26187291.

7. Choi C, Ahn S, Min SI, Ahn M, Ha J, Yoon HJ, et al. Nationwide epidemiologic study of abdominal aortic aneurysms in Korea: a cross-sectional study using national health insurance review and assessment service data. Vasc Specialist Int. 2019; 35:193–201. PMID: 31915663.

8. de Guerre LE, Dansey KD, Patel PB, O’Donnell TF, Zettervall SL, van Herwaarden JA, et al. Not all risk scores are created equal: a comparison of risk scores for abdominal aortic aneurysm repair in administrative data and quality improvement registries. J Vasc Surg. 2021; 74:1874–1884. PMID: 34302935.

9. Giles KA, Stone DH, Beck AW, Huber TS, Upchurch GR Jr, Arnaoutakis DJ, et al. Association of hospital volume with patient selection, risk of complications, and mortality from failure to rescue after open abdominal aortic aneurysm repair. J Vasc Surg. 2020; 72:1681–1690. PMID: 32294505.

10. Arnaoutakis DJ, Scali ST, Neal D, Giles KA, Huber TS, Powell RJ, et al. Surgeon experience association with patient selection and outcomes after open abdominal aortic aneurysm repair. J Vasc Surg. 2020; 72:1325–1336. PMID: 32115318.

11. Powell JT, Sweeting MJ, Ulug P, Blankensteijn JD, Lederle FA, Becquemin JP, et al. Meta-analysis of individual-patient data from EVAR-1, DREAM, OVER and ACE trials comparing outcomes of endovascular or open repair for abdominal aortic aneurysm over 5 years. Br J Surg. 2017; 104:166–178. PMID: 28160528.

12. Hwang D, Kim HK, Huh S. Incidence and risk factors for sac expansion after endovascular aneurysm repair of abdominal aortic aneurysms. Vasc Specialist Int. 2021; 37:34. PMID: 34753833.

13. Min SK. Controversies in the Treatment of Abdominal Aortic Aneurysm (I): Is the NICE Guideline Nice to Patients? Vasc Specialist Int. 2021; 37:39. PMID: 34961750.

14. McNally MM, Agle SC, Parker FM, Bogey WM, Powell CS, Stoner MC. Preoperative statin therapy is associated with improved outcomes and resource utilization in patients undergoing aortic aneurysm repair. J Vasc Surg. 2010; 51:1390–1396. PMID: 20382494.

15. Wohlauer M, Brier C, Kuramochi Y, Eagleton M. Preoperative hypoalbuminemia is a risk factor for early and late mortality in patients undergoing endovascular juxtarenal and thoracoabdominal aortic aneurysm repair. Ann Vasc Surg. 2017; 42:198–204. PMID: 28359793.

16. Lindström I, Khan N, Vänttinen T, Peltokangas M, Sillanpää N, Oksala N. Psoas muscle area and quality are independent predictors of survival in patients treated for abdominal aortic aneurysms. Ann Vasc Surg. 2019; 56:183–193. PMID: 30476615.

17. Li B, Aljabri B, Verma R, Beaton D, Eisenberg N, Lee DS, et al. Using machine learning to predict outcomes following open abdominal aortic aneurysm repair. J Vasc Surg. 2023; 78:1426–1438. PMID: 37634621.

18. Li B, Aljabri B, Verma R, Beaton D, Eisenberg N, Lee DS, et al. Machine learning to predict outcomes following endovascular abdominal aortic aneurysm repair. Br J Surg. 2023; 110:1840–1849. PMID: 37710397.