Systematic evaluation of machine learning models for postoperative surgical site infection prediction.
<h4>Background</h4>Surgical site infections (SSIs) lead to increased mortality and morbidity, as well as increased healthcare costs. Multiple models for the prediction of this serious surgical complication have been developed, with an increasing use of machine learning (ML) tools.<h4&...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2024-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://doi.org/10.1371/journal.pone.0312968 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846119355314601984 |
|---|---|
| author | Anna M van Boekel Siri L van der Meijden Sesmu M Arbous Rob G H H Nelissen Karin E Veldkamp Emma B Nieswaag Kim F T Jochems Jeroen Holtz Annekee van IJlzinga Veenstra Jeroen Reijman Ype de Jong Harry van Goor Maryse A Wiewel Jan W Schoones Bart F Geerts Mark G J de Boer |
| author_facet | Anna M van Boekel Siri L van der Meijden Sesmu M Arbous Rob G H H Nelissen Karin E Veldkamp Emma B Nieswaag Kim F T Jochems Jeroen Holtz Annekee van IJlzinga Veenstra Jeroen Reijman Ype de Jong Harry van Goor Maryse A Wiewel Jan W Schoones Bart F Geerts Mark G J de Boer |
| author_sort | Anna M van Boekel |
| collection | DOAJ |
| description | <h4>Background</h4>Surgical site infections (SSIs) lead to increased mortality and morbidity, as well as increased healthcare costs. Multiple models for the prediction of this serious surgical complication have been developed, with an increasing use of machine learning (ML) tools.<h4>Objective</h4>The aim of this systematic review was to assess the performance as well as the methodological quality of validated ML models for the prediction of SSIs.<h4>Methods</h4>A systematic search in PubMed, Embase and the Cochrane library was performed from inception until July 2023. Exclusion criteria were the absence of reported model validation, SSIs as part of a composite adverse outcome, and pediatric populations. ML performance measures were evaluated, and ML performances were compared to regression-based methods for studies that reported both methods. Risk of bias (ROB) of the studies was assessed using the Prediction model Risk of Bias Assessment Tool.<h4>Results</h4>Of the 4,377 studies screened, 24 were included in this review, describing 85 ML models. Most models were only internally validated (81%). The C-statistic was the most used performance measure (reported in 96% of the studies) and only two studies reported calibration metrics. A total of 116 different predictors were described, of which age, steroid use, sex, diabetes, and smoking were most frequently (100% to 75%) incorporated. Thirteen studies compared ML models to regression-based models and showed a similar performance of both modelling methods. For all included studies, the overall ROB was high or unclear.<h4>Conclusions</h4>A multitude of ML models for the prediction of SSIs are available, with large variability in performance. However, most models lacked external validation, performance was reported limitedly, and the risk of bias was high. In studies describing both ML models and regression-based models, one modelling method did not outperform the other. |
| format | Article |
| id | doaj-art-c61c08b7b37e4fa4a2aa32b3df1a1e40 |
| institution | Kabale University |
| issn | 1932-6203 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj-art-c61c08b7b37e4fa4a2aa32b3df1a1e402024-12-17T05:31:43ZengPublic Library of Science (PLoS)PLoS ONE1932-62032024-01-011912e031296810.1371/journal.pone.0312968Systematic evaluation of machine learning models for postoperative surgical site infection prediction.Anna M van BoekelSiri L van der MeijdenSesmu M ArbousRob G H H NelissenKarin E VeldkampEmma B NieswaagKim F T JochemsJeroen HoltzAnnekee van IJlzinga VeenstraJeroen ReijmanYpe de JongHarry van GoorMaryse A WiewelJan W SchoonesBart F GeertsMark G J de Boer<h4>Background</h4>Surgical site infections (SSIs) lead to increased mortality and morbidity, as well as increased healthcare costs. Multiple models for the prediction of this serious surgical complication have been developed, with an increasing use of machine learning (ML) tools.<h4>Objective</h4>The aim of this systematic review was to assess the performance as well as the methodological quality of validated ML models for the prediction of SSIs.<h4>Methods</h4>A systematic search in PubMed, Embase and the Cochrane library was performed from inception until July 2023. Exclusion criteria were the absence of reported model validation, SSIs as part of a composite adverse outcome, and pediatric populations. ML performance measures were evaluated, and ML performances were compared to regression-based methods for studies that reported both methods. Risk of bias (ROB) of the studies was assessed using the Prediction model Risk of Bias Assessment Tool.<h4>Results</h4>Of the 4,377 studies screened, 24 were included in this review, describing 85 ML models. Most models were only internally validated (81%). The C-statistic was the most used performance measure (reported in 96% of the studies) and only two studies reported calibration metrics. A total of 116 different predictors were described, of which age, steroid use, sex, diabetes, and smoking were most frequently (100% to 75%) incorporated. Thirteen studies compared ML models to regression-based models and showed a similar performance of both modelling methods. For all included studies, the overall ROB was high or unclear.<h4>Conclusions</h4>A multitude of ML models for the prediction of SSIs are available, with large variability in performance. However, most models lacked external validation, performance was reported limitedly, and the risk of bias was high. In studies describing both ML models and regression-based models, one modelling method did not outperform the other.https://doi.org/10.1371/journal.pone.0312968 |
| spellingShingle | Anna M van Boekel Siri L van der Meijden Sesmu M Arbous Rob G H H Nelissen Karin E Veldkamp Emma B Nieswaag Kim F T Jochems Jeroen Holtz Annekee van IJlzinga Veenstra Jeroen Reijman Ype de Jong Harry van Goor Maryse A Wiewel Jan W Schoones Bart F Geerts Mark G J de Boer Systematic evaluation of machine learning models for postoperative surgical site infection prediction. PLoS ONE |
| title | Systematic evaluation of machine learning models for postoperative surgical site infection prediction. |
| title_full | Systematic evaluation of machine learning models for postoperative surgical site infection prediction. |
| title_fullStr | Systematic evaluation of machine learning models for postoperative surgical site infection prediction. |
| title_full_unstemmed | Systematic evaluation of machine learning models for postoperative surgical site infection prediction. |
| title_short | Systematic evaluation of machine learning models for postoperative surgical site infection prediction. |
| title_sort | systematic evaluation of machine learning models for postoperative surgical site infection prediction |
| url | https://doi.org/10.1371/journal.pone.0312968 |
| work_keys_str_mv | AT annamvanboekel systematicevaluationofmachinelearningmodelsforpostoperativesurgicalsiteinfectionprediction AT sirilvandermeijden systematicevaluationofmachinelearningmodelsforpostoperativesurgicalsiteinfectionprediction AT sesmumarbous systematicevaluationofmachinelearningmodelsforpostoperativesurgicalsiteinfectionprediction AT robghhnelissen systematicevaluationofmachinelearningmodelsforpostoperativesurgicalsiteinfectionprediction AT karineveldkamp systematicevaluationofmachinelearningmodelsforpostoperativesurgicalsiteinfectionprediction AT emmabnieswaag systematicevaluationofmachinelearningmodelsforpostoperativesurgicalsiteinfectionprediction AT kimftjochems systematicevaluationofmachinelearningmodelsforpostoperativesurgicalsiteinfectionprediction AT jeroenholtz systematicevaluationofmachinelearningmodelsforpostoperativesurgicalsiteinfectionprediction AT annekeevanijlzingaveenstra systematicevaluationofmachinelearningmodelsforpostoperativesurgicalsiteinfectionprediction AT jeroenreijman systematicevaluationofmachinelearningmodelsforpostoperativesurgicalsiteinfectionprediction AT ypedejong systematicevaluationofmachinelearningmodelsforpostoperativesurgicalsiteinfectionprediction AT harryvangoor systematicevaluationofmachinelearningmodelsforpostoperativesurgicalsiteinfectionprediction AT maryseawiewel systematicevaluationofmachinelearningmodelsforpostoperativesurgicalsiteinfectionprediction AT janwschoones systematicevaluationofmachinelearningmodelsforpostoperativesurgicalsiteinfectionprediction AT bartfgeerts systematicevaluationofmachinelearningmodelsforpostoperativesurgicalsiteinfectionprediction AT markgjdeboer systematicevaluationofmachinelearningmodelsforpostoperativesurgicalsiteinfectionprediction |