Advancing a machine learning-based decision support tool for pre-hospital assessment of dyspnoea by emergency medical service clinicians: a retrospective observational study
Abstract Background In Sweden with about 10 million inhabitants, there are about one million primary ambulance missions every year. Among them, around 10% are assessed by Emergency Medical Service (EMS) clinicians with the primary symptom of dyspnoea. The risk of death among these patients has been...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2025-01-01
|
| Series: | BMC Emergency Medicine |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12873-024-01166-9 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841544939279745024 |
|---|---|
| author | Wivica Kauppi Henrik Imberg Johan Herlitz Oskar Molin Christer Axelsson Carl Magnusson |
| author_facet | Wivica Kauppi Henrik Imberg Johan Herlitz Oskar Molin Christer Axelsson Carl Magnusson |
| author_sort | Wivica Kauppi |
| collection | DOAJ |
| description | Abstract Background In Sweden with about 10 million inhabitants, there are about one million primary ambulance missions every year. Among them, around 10% are assessed by Emergency Medical Service (EMS) clinicians with the primary symptom of dyspnoea. The risk of death among these patients has been reported to be remarkably high, at 11,1% and 13,2%. The aim was to develop a Machine Learning (ML) model to provide support in assessing patients in pre-hospital settings and to compare them with established triage tools. Methods This was a retrospective observational study including 6,354 patients who called the Swedish emergency telephone number (112) between January and December 2017. Patients presenting with the main symptom of dyspnoea were included which were recruited from two EMS organisations in Göteborg and Södra Älvsborg. Serious Adverse Event (SAE) was used as outcome, defined as any of the following:1) death within 30 days after call for an ambulance, 2) a final diagnosis defined as time-sensitive, 3) admitted to intensive care unit, or 4) readmission within 72 h and admitted to hospital receiving a final time-sensitive diagnosis. Logistic regression, LASSO logistic regression and gradient boosting were compared to the Rapid Emergency Triage and Treatment System for Adults (RETTS-A) and National Early Warning Score2 (NEWS2) with respect to discrimination and calibration of predictions. Eighty percent (80%) of the data was used for model development and 20% for model validation. Results All ML models showed better performance than RETTS-A and NEWS2 with respect to all evaluated performance metrics. The gradient boosting algorithm had the overall best performance, with excellent calibration of the predictions, and consistently showed higher sensitivity to detect SAE than the other methods. The ROC AUC on test data increased from 0.73 (95% CI 0.70–0.76) with RETTS-A to 0.81 (95% CI 0.78–0.84) using gradient boosting. Conclusions Among 6,354 ambulance missions caused by patients suffering from dyspnoea, an ML method using gradient boosting demonstrated excellent performance for predicting SAE, with substantial improvement over the more established methods RETTS-A and NEWS2. |
| format | Article |
| id | doaj-art-d9a872b92d314e7e8146f9e31d920814 |
| institution | Kabale University |
| issn | 1471-227X |
| language | English |
| publishDate | 2025-01-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Emergency Medicine |
| spelling | doaj-art-d9a872b92d314e7e8146f9e31d9208142025-01-12T12:10:36ZengBMCBMC Emergency Medicine1471-227X2025-01-0125111210.1186/s12873-024-01166-9Advancing a machine learning-based decision support tool for pre-hospital assessment of dyspnoea by emergency medical service clinicians: a retrospective observational studyWivica Kauppi0Henrik Imberg1Johan Herlitz2Oskar Molin3Christer Axelsson4Carl Magnusson5PreHospen- Centre for Prehospital Research, Faculty of Caring Science, Work Life and Social Welfare, University of BoråsStatistiska Konsultgruppen SwedenPreHospen- Centre for Prehospital Research, Faculty of Caring Science, Work Life and Social Welfare, University of BoråsStatistiska Konsultgruppen SwedenPreHospen- Centre for Prehospital Research, Faculty of Caring Science, Work Life and Social Welfare, University of BoråsPreHospen- Centre for Prehospital Research, Faculty of Caring Science, Work Life and Social Welfare, University of BoråsAbstract Background In Sweden with about 10 million inhabitants, there are about one million primary ambulance missions every year. Among them, around 10% are assessed by Emergency Medical Service (EMS) clinicians with the primary symptom of dyspnoea. The risk of death among these patients has been reported to be remarkably high, at 11,1% and 13,2%. The aim was to develop a Machine Learning (ML) model to provide support in assessing patients in pre-hospital settings and to compare them with established triage tools. Methods This was a retrospective observational study including 6,354 patients who called the Swedish emergency telephone number (112) between January and December 2017. Patients presenting with the main symptom of dyspnoea were included which were recruited from two EMS organisations in Göteborg and Södra Älvsborg. Serious Adverse Event (SAE) was used as outcome, defined as any of the following:1) death within 30 days after call for an ambulance, 2) a final diagnosis defined as time-sensitive, 3) admitted to intensive care unit, or 4) readmission within 72 h and admitted to hospital receiving a final time-sensitive diagnosis. Logistic regression, LASSO logistic regression and gradient boosting were compared to the Rapid Emergency Triage and Treatment System for Adults (RETTS-A) and National Early Warning Score2 (NEWS2) with respect to discrimination and calibration of predictions. Eighty percent (80%) of the data was used for model development and 20% for model validation. Results All ML models showed better performance than RETTS-A and NEWS2 with respect to all evaluated performance metrics. The gradient boosting algorithm had the overall best performance, with excellent calibration of the predictions, and consistently showed higher sensitivity to detect SAE than the other methods. The ROC AUC on test data increased from 0.73 (95% CI 0.70–0.76) with RETTS-A to 0.81 (95% CI 0.78–0.84) using gradient boosting. Conclusions Among 6,354 ambulance missions caused by patients suffering from dyspnoea, an ML method using gradient boosting demonstrated excellent performance for predicting SAE, with substantial improvement over the more established methods RETTS-A and NEWS2.https://doi.org/10.1186/s12873-024-01166-9DyspnoeaSerious adverse eventPrehospitalAmbulanceEmergency medical servicesMachine learning |
| spellingShingle | Wivica Kauppi Henrik Imberg Johan Herlitz Oskar Molin Christer Axelsson Carl Magnusson Advancing a machine learning-based decision support tool for pre-hospital assessment of dyspnoea by emergency medical service clinicians: a retrospective observational study BMC Emergency Medicine Dyspnoea Serious adverse event Prehospital Ambulance Emergency medical services Machine learning |
| title | Advancing a machine learning-based decision support tool for pre-hospital assessment of dyspnoea by emergency medical service clinicians: a retrospective observational study |
| title_full | Advancing a machine learning-based decision support tool for pre-hospital assessment of dyspnoea by emergency medical service clinicians: a retrospective observational study |
| title_fullStr | Advancing a machine learning-based decision support tool for pre-hospital assessment of dyspnoea by emergency medical service clinicians: a retrospective observational study |
| title_full_unstemmed | Advancing a machine learning-based decision support tool for pre-hospital assessment of dyspnoea by emergency medical service clinicians: a retrospective observational study |
| title_short | Advancing a machine learning-based decision support tool for pre-hospital assessment of dyspnoea by emergency medical service clinicians: a retrospective observational study |
| title_sort | advancing a machine learning based decision support tool for pre hospital assessment of dyspnoea by emergency medical service clinicians a retrospective observational study |
| topic | Dyspnoea Serious adverse event Prehospital Ambulance Emergency medical services Machine learning |
| url | https://doi.org/10.1186/s12873-024-01166-9 |
| work_keys_str_mv | AT wivicakauppi advancingamachinelearningbaseddecisionsupporttoolforprehospitalassessmentofdyspnoeabyemergencymedicalservicecliniciansaretrospectiveobservationalstudy AT henrikimberg advancingamachinelearningbaseddecisionsupporttoolforprehospitalassessmentofdyspnoeabyemergencymedicalservicecliniciansaretrospectiveobservationalstudy AT johanherlitz advancingamachinelearningbaseddecisionsupporttoolforprehospitalassessmentofdyspnoeabyemergencymedicalservicecliniciansaretrospectiveobservationalstudy AT oskarmolin advancingamachinelearningbaseddecisionsupporttoolforprehospitalassessmentofdyspnoeabyemergencymedicalservicecliniciansaretrospectiveobservationalstudy AT christeraxelsson advancingamachinelearningbaseddecisionsupporttoolforprehospitalassessmentofdyspnoeabyemergencymedicalservicecliniciansaretrospectiveobservationalstudy AT carlmagnusson advancingamachinelearningbaseddecisionsupporttoolforprehospitalassessmentofdyspnoeabyemergencymedicalservicecliniciansaretrospectiveobservationalstudy |