Register
Forgot your password?

The effectiveness of machine learning in predicting early postoperative death after coronary bypass surgery

Authors: Golukhova E.Z., Keren M.A., Zavalikhina T.V., Bulaeva N.I., Grechishnikova D.A., Sigaev I.Yu., Yakhyaeva K.B.

Company: Bakoulev National Medical Research Center for Cardiovascular Surgery, Moscow, Russian Federation

For correspondence:  Sign in or register.

Type:  Original articles


DOI: https://doi.org/10.24022/1997-3187-2023-17-1-77-93

For citation: Golukhova E.Z., Keren M.A., Zavalikhina T.V., Bulaeva N.I., Grechishnikova D.A., Sigaev I.Yu., Yakhyaeva K.B. The effectiveness of machine learning models in predicting early postoperative death after coronary bypass surgery. Creative Cardiology. 2023; 17 (1): 77–93 (in Russ.). DOI: 10.24022/1997-3187-2023-17-1-77-93

Received / Accepted:  09.01.2023 / 13.03.2023

Keywords: machine learning coronary artery bypass grafting death prediction

Download
Full text:  

 

Abstract

Objective. To develop and evaluate the effectiveness of models for predicting mortality after coronary bypass surgery versus EuroSCORE II.

Material and methods. The analysis included anonymised medical data of 2,826 patients with stable IHD after coronary artery bypass grafting (CABG). Mortality in the group was 1.45% (n = 41). The target feature for prediction was death from any cause after CABG. To predict death after CABG, logistic regression (LR) and gradient boosting methods were used: LightGBM, XGBoost, CatBoost and ensemble model (EM), which consisted of all four developed models. The results of the models were compared with risk model EuroSCORE II. Precision, recall, F1-score and AUC were used to evaluate the effectiveness.

Results. The LR model had the highest ability to predict the outcomes of a positive class among the general class (recall – 0.88), but had the lowest accuracy value (precision – 0.03). Gradient boosting models (LightGBM, XGBoost, Catboost and EM), had higher accuracy precision and acceptable recall and F1-score values. EM had the highest predictive efficiency when comparing models (recall – 0.50, precision – 0.67, F1-measure – 0.57, AUC – 0.85). The EuroSCORE II did not show sufficient effectiveness in predicting the risk of death after CABG in the analyzed patients (recall – 0.143, precision – 0.125, F1-measure – 0.133, AUC – 0.47). The best discrimination indicators were obtained for the LightGBM (AUC – 0.84), Catboost (AUC – 0.87) and EM (AUC – 0.85) models.

Conclusion. We have developed models for predicting death after coronary bypass surgery using machine-learning methods (LR, LightGBM, XGBoost, CatBoost and EM). The quality metrics of models developed using machine learning methods, especially gradient boosting methods, exceed the predictive capabilities of the EuroSCORE II.

References

  1. ESC/EACTS recommendations on myocardial revascularization 2018. Russian Journal of Cardiology. 2019; 8: 151–226. DOI: 10.15829/1560- 4071-2019-8-151-226 (in Russ.).
  2. Allyn J., Allou N., Augustin P., Philip I., Martinet O., Belghiti M. et al. Comparison of a machine learning model with EuroSCORE II in predicting mortality after elective cardiac surgery: a decision curve analysis. PLoS One. 2017; 12 (1): e0169772. DOI: 10.1371/journal.pone.0169772
  3. Molina R.S., Molina-Rodríguez M.A., Rincón F.M., Maldonado J.D. Cardiac operative risk in Latin America: a comparison of machine learning models vs EuroSCORE-II. Ann. Thorac. Surg. 2022; 113 (1): 92–9. DOI: 10.1016/j.athoracsur.2021.02.052
  4. Deo R.C. Machine learning in medicine. Circulation. 2015; 132 (20): 1920–30. DOI: 10.1161/CIRCULATIONAHA.115.001593
  5. Arfat Y., Mittone G., Esposito R., Cantalupo B., de Ferrari G.M., Aldinucci M. Machine learning for cardiology. Minerva Cardiol. Angiol. 2022; 70 (1): 75–91. DOI: 10.23736/S2724-5683.21. 05709-4
  6. Penny-Dimri J.C., Bergmeir C., Perry L., Hayes L., Bellomo R., Smith J.A. Machine learning to predict adverse outcomes after cardiac surgery: a systematic review and meta-analysis. J. Card. Surg. 2022; 37 (11): 3838–45. DOI: 10.1111/jocs.16842
  7. Benedetto U., Dimagli A., Sinha S., Cocomello L., Gibbison B., Caputo M. et al. Machine learning improves mortality risk prediction after cardiac surgery: systematic review and meta-analysis. J. Thorac. Cardiovasc. Surg. 2022; 163 (6): 2075–87.e9. DOI: 10.1016/j.jtcvs.2020.07.105
  8. Nashef S.A., Roques F., Sharples L.D., Nilsson J., Smith C., Goldstone A.R. et al. EuroSCORE II. Eur. J. Cardiothorac. Surg. 2012; 41 (4): 734–44; discussion 744–5. DOI: 10.1093/ejcts/ezs043
  9. Shahian D.M., O’Brien S.M., Filardo G., Ferraris V.A., Haan C.K., Rich J.B. et al. Society of Thoracic Surgeons Quality Measurement Task Force. The Society of Thoracic Surgeons 2008 cardiac surgery risk models: part 1 – coronary artery bypass grafting surgery. Ann. Thorac. Surg. 2009; 88 (1 Suppl.): S2–22. DOI: 10.1016/j.athoracsur. 2009.05.053
  10. Hastie T., Tibshirani R., Friedman J. The elements of statistical learning. New York; Springer: 2009. https://link.springer.com/book/10.1007/978-0- 387-84858-7 (дата обращения 04.03.2023 /accessed March 04, 2023).
  11. Verma A., Sanaiha Y., Hadaya J., Maltagliati A.J., Tran Z., Ramezani R. et al. University of California Cardiac Surgery Consortium. Parsimonious machine learning models to predict resource use in cardiac surgery across a statewide collaborative. JTCVS Open. 2022; 11: 214–28. DOI: 10.1016/j.xjon.2022.04.017
  12. Lee H.C., Yoon H.K., Nam K., Cho Y.J., Kim T.K., Kim W.H. et al. Derivation and validation of machine learning approaches to predict acute kidney injury after cardiac surgery. J. Clin. Med. 2018; 7 (10): 322. DOI: 10.3390/jcm7100322
  13. Ivanov J., Tu J.V., Naylor C.D. Ready-made, recalibrated, or Remodeled? Issues in the use of risk indexes for assessing mortality after coronary artery bypass graft surgery. Circulation. 1999; 99 (16): 2098–104. DOI: 10.1161/01.cir.99.16.2098
  14. Silverborn M., Nielsen S., Karlsson M. The performance of EuroSCORE II in CABG patients in relation to sex, age, and surgical risk: a nationwide study in 14,118 patients. J. Cardiothorac. Surg. 2023; 18 (1): 40. DOI: 10.1186/s13019-023-02141-4
  15. Shahian D.M., Blackstone E.H., Edwards F.H., Grover F.L., Grunkemeier G.L., Naftel D.C. et al. STS workforce on evidence-based surgery. Cardiac surgery risk models: a position article. Ann. Thorac. Surg. 2004; 78 (5): 1868–77. DOI: 10.1016/j.athoracsur.2004.05.054 16. Karim M.N., Reid C.M., Cochrane A., Tran L., Alramadan M., Hossain M.N. Mortality risk prediction models for coronary artery bypass graft surgery: current scenario and future direction. J. Cardiovasc. Surg. (Torino). 2017; 58 (6): 931–42. DOI: 10.23736/S0021-9509.17.09965-7 17. Harrell F.E., Jr. Regression modeling strategies. With applications to linear models, logistic regression, and survival analysis. New York; Springer: 2001. https://link.springer.com/book/10.1007/978-1-4757-3462-1 (дата обращения 10.03.2023 /accessed March 10, 2023).
  16. Lawton J.S., Tamis-Holland J.E., Bangalore S., Bates E.R., Beckie T.M., Bischoff J.M. et al. 2021 ACC/AHA/SCAI Guideline for Coronary Artery Revascularization. Circulation. 2022; 145 (3): e4–17. DOI: 10.1161/CIR.0000000000001039

About Authors

  • Elena Z. Golukhova, Dr. Med. Sci., Professor, Academician of RAS, Director; ORCID
  • Milena A. Keren, Dr. Med. Sci., Senior Researcher; ORCID
  • Tatyana V. Zavalikhina, Cand. Med. Sci., Deputy Chief Physician; ORCID
  • Naida I. Bulaeva, Cand. Biol. Sci., Senior Researcher, Cardiologist, Head of Department; ORCID
  • Darya A. Grechishnikova, Cand. Phys. Tech. Sci.; ORCID
  • Igor Yu. Sigaev, Dr. Med. Sci., Professor, Head of Department; ORCID
  • Kumushdzhan B. Yakhyaeva, Cardiologist; ORCID

Chief Editor

Leo A. Bockeria, MD, PhD, DSc, Professor, Academician of Russian Academy of Sciences, President of Bakoulev National Medical Research Center for Cardiovascular Surgery


Sort by