External validation of an artificial intelligence model using clinical variables, including ICD-10 codes, for predicting in-hospital mortality among trauma patients: a multicenter retrospective cohort study
Abstract Artificial intelligence (AI) is being increasingly applied in healthcare to improve patient care and clinical outcomes. We previously developed an AI model using ICD-10 (International Classification of Diseases, Tenth Revision) codes with other clinical variables to predict in-hospital mort...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-01-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-025-85420-5 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841544845678608384 |
|---|---|
| author | Seungseok Lee Do Wan Kim Na-eun Oh Hayeon Lee Seoyoung Park Dong Keon Yon Wu Seong Kang Jinseok Lee |
| author_facet | Seungseok Lee Do Wan Kim Na-eun Oh Hayeon Lee Seoyoung Park Dong Keon Yon Wu Seong Kang Jinseok Lee |
| author_sort | Seungseok Lee |
| collection | DOAJ |
| description | Abstract Artificial intelligence (AI) is being increasingly applied in healthcare to improve patient care and clinical outcomes. We previously developed an AI model using ICD-10 (International Classification of Diseases, Tenth Revision) codes with other clinical variables to predict in-hospital mortality among trauma patients from a nationwide database. This study aimed to externally validate the performance of the AI model. Validation was conducted using a multicenter retrospective cohort study design, analyzing patient data from January 2020 to December 2021. The study included trauma patients based on specific ICD-10 codes, with other clinical variables. The performance of the AI model was evaluated against conventional metrics, including the ISS, and the ICISS (ICD-based ISS), using sensitivity, specificity, accuracy, balanced accuracy, precision, F1-score, and area under the receiver operating characteristic curve (AUROC) analyses. Data from 4,439 patients were analyzed. The AI model demonstrated high overall performance, achieving an AUROC of 0.9448 and a balanced accuracy of 85.08%, thereby outperforming traditional scoring systems such as ISS, or ICISS. Furthermore, the model accurately predicted mortality across datasets from each hospital (AUROCs of 0.9234 and 0.9653, respectively) despite significant differences in hospital characteristics. In the subset of patients with ISS < 9, the model showed a robust AUROC of 0.9043, indicating its effectiveness in predicting mortality, even in cases with lower-severity injuries. For patients with ISSs ≥ 9, the model maintained high sensitivity (93.60%) and balanced accuracy (77.08%), proving its reliability in more severe injury cases. External validation demonstrated the AI model’s high predictive accuracy and reliability in assessing in-hospital mortality risk among trauma patients across different injury severities and heterogeneous cohorts. These findings support the model’s potential integration into emergency departments and offer a significant tool for enhancing patient triage and treatment protocols. |
| format | Article |
| id | doaj-art-c4bb3bf0714a4fd5932c52b45bc6916e |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-c4bb3bf0714a4fd5932c52b45bc6916e2025-01-12T12:15:21ZengNature PortfolioScientific Reports2045-23222025-01-0115111510.1038/s41598-025-85420-5External validation of an artificial intelligence model using clinical variables, including ICD-10 codes, for predicting in-hospital mortality among trauma patients: a multicenter retrospective cohort studySeungseok Lee0Do Wan Kim1Na-eun Oh2Hayeon Lee3Seoyoung Park4Dong Keon Yon5Wu Seong Kang6Jinseok Lee7Department of Biomedical Engineering, Kyung Hee UniversityDepartment of Thoracic and Cardiovascular Surgery, Chonnam National University Hospital, Chonnam National University Medical SchoolDepartment of Biomedical Engineering, Kyung Hee UniversityDepartment of Electronics and Information Convergence Engineering, Kyung Hee UniversityCenter for Digital Health, Medical Science Research Institute, Kyung Hee University College of MedicineCenter for Digital Health, Medical Science Research Institute, Kyung Hee University College of MedicineDepartment of Trauma Surgery, Jeju Regional Trauma Center, Cheju Halla General HospitalDepartment of Biomedical Engineering, Kyung Hee UniversityAbstract Artificial intelligence (AI) is being increasingly applied in healthcare to improve patient care and clinical outcomes. We previously developed an AI model using ICD-10 (International Classification of Diseases, Tenth Revision) codes with other clinical variables to predict in-hospital mortality among trauma patients from a nationwide database. This study aimed to externally validate the performance of the AI model. Validation was conducted using a multicenter retrospective cohort study design, analyzing patient data from January 2020 to December 2021. The study included trauma patients based on specific ICD-10 codes, with other clinical variables. The performance of the AI model was evaluated against conventional metrics, including the ISS, and the ICISS (ICD-based ISS), using sensitivity, specificity, accuracy, balanced accuracy, precision, F1-score, and area under the receiver operating characteristic curve (AUROC) analyses. Data from 4,439 patients were analyzed. The AI model demonstrated high overall performance, achieving an AUROC of 0.9448 and a balanced accuracy of 85.08%, thereby outperforming traditional scoring systems such as ISS, or ICISS. Furthermore, the model accurately predicted mortality across datasets from each hospital (AUROCs of 0.9234 and 0.9653, respectively) despite significant differences in hospital characteristics. In the subset of patients with ISS < 9, the model showed a robust AUROC of 0.9043, indicating its effectiveness in predicting mortality, even in cases with lower-severity injuries. For patients with ISSs ≥ 9, the model maintained high sensitivity (93.60%) and balanced accuracy (77.08%), proving its reliability in more severe injury cases. External validation demonstrated the AI model’s high predictive accuracy and reliability in assessing in-hospital mortality risk among trauma patients across different injury severities and heterogeneous cohorts. These findings support the model’s potential integration into emergency departments and offer a significant tool for enhancing patient triage and treatment protocols.https://doi.org/10.1038/s41598-025-85420-5Artificial IntelligenceIn-Hospital mortalityTrauma patientsICD-10External validationInjury Severity score |
| spellingShingle | Seungseok Lee Do Wan Kim Na-eun Oh Hayeon Lee Seoyoung Park Dong Keon Yon Wu Seong Kang Jinseok Lee External validation of an artificial intelligence model using clinical variables, including ICD-10 codes, for predicting in-hospital mortality among trauma patients: a multicenter retrospective cohort study Scientific Reports Artificial Intelligence In-Hospital mortality Trauma patients ICD-10 External validation Injury Severity score |
| title | External validation of an artificial intelligence model using clinical variables, including ICD-10 codes, for predicting in-hospital mortality among trauma patients: a multicenter retrospective cohort study |
| title_full | External validation of an artificial intelligence model using clinical variables, including ICD-10 codes, for predicting in-hospital mortality among trauma patients: a multicenter retrospective cohort study |
| title_fullStr | External validation of an artificial intelligence model using clinical variables, including ICD-10 codes, for predicting in-hospital mortality among trauma patients: a multicenter retrospective cohort study |
| title_full_unstemmed | External validation of an artificial intelligence model using clinical variables, including ICD-10 codes, for predicting in-hospital mortality among trauma patients: a multicenter retrospective cohort study |
| title_short | External validation of an artificial intelligence model using clinical variables, including ICD-10 codes, for predicting in-hospital mortality among trauma patients: a multicenter retrospective cohort study |
| title_sort | external validation of an artificial intelligence model using clinical variables including icd 10 codes for predicting in hospital mortality among trauma patients a multicenter retrospective cohort study |
| topic | Artificial Intelligence In-Hospital mortality Trauma patients ICD-10 External validation Injury Severity score |
| url | https://doi.org/10.1038/s41598-025-85420-5 |
| work_keys_str_mv | AT seungseoklee externalvalidationofanartificialintelligencemodelusingclinicalvariablesincludingicd10codesforpredictinginhospitalmortalityamongtraumapatientsamulticenterretrospectivecohortstudy AT dowankim externalvalidationofanartificialintelligencemodelusingclinicalvariablesincludingicd10codesforpredictinginhospitalmortalityamongtraumapatientsamulticenterretrospectivecohortstudy AT naeunoh externalvalidationofanartificialintelligencemodelusingclinicalvariablesincludingicd10codesforpredictinginhospitalmortalityamongtraumapatientsamulticenterretrospectivecohortstudy AT hayeonlee externalvalidationofanartificialintelligencemodelusingclinicalvariablesincludingicd10codesforpredictinginhospitalmortalityamongtraumapatientsamulticenterretrospectivecohortstudy AT seoyoungpark externalvalidationofanartificialintelligencemodelusingclinicalvariablesincludingicd10codesforpredictinginhospitalmortalityamongtraumapatientsamulticenterretrospectivecohortstudy AT dongkeonyon externalvalidationofanartificialintelligencemodelusingclinicalvariablesincludingicd10codesforpredictinginhospitalmortalityamongtraumapatientsamulticenterretrospectivecohortstudy AT wuseongkang externalvalidationofanartificialintelligencemodelusingclinicalvariablesincludingicd10codesforpredictinginhospitalmortalityamongtraumapatientsamulticenterretrospectivecohortstudy AT jinseoklee externalvalidationofanartificialintelligencemodelusingclinicalvariablesincludingicd10codesforpredictinginhospitalmortalityamongtraumapatientsamulticenterretrospectivecohortstudy |