Beyond the ROC Curve: Activity Monitoring to Evaluate Deep Learning Models in Clinical Settings
We evaluated ‘VITALCARE-SEPS’, a deep learning model for sepsis prediction, using the activity monitoring operator characteristics curve with two different scoring algorithms. This evaluation is crucial as the AMOC curve addresses the time-dependent nature of predictions, providing a more nuanced p...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Iuliu Hatieganu University of Medicine and Pharmacy, Cluj-Napoca
2024-11-01
|
| Series: | Applied Medical Informatics |
| Subjects: | |
| Online Access: | https://ami.info.umfcluj.ro/index.php/AMI/article/view/1074 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841558954218356736 |
|---|---|
| author | Hyunwoo CHOO Kyung Hyun LEE Sungsoo HONG Sungjun HONG Ki-Byung LEE Chang Youl LEE |
| author_facet | Hyunwoo CHOO Kyung Hyun LEE Sungsoo HONG Sungjun HONG Ki-Byung LEE Chang Youl LEE |
| author_sort | Hyunwoo CHOO |
| collection | DOAJ |
| description |
We evaluated ‘VITALCARE-SEPS’, a deep learning model for sepsis prediction, using the activity monitoring operator characteristics curve with two different scoring algorithms. This evaluation is crucial as the AMOC curve addresses the time-dependent nature of predictions, providing a more nuanced performance assessment than traditional ROC metrics. Our findings demonstrate that the AMOC curve significantly enhances the evaluation of time-series predictions, enabling more accurate and continuous performance monitoring of machine learning models in clinical settings. This approach can improve model deployment and ultimately lead to better patient outcomes in healthcare.
|
| format | Article |
| id | doaj-art-f1e7b0166a5c426b98ba5ee3607454fb |
| institution | Kabale University |
| issn | 2067-7855 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Iuliu Hatieganu University of Medicine and Pharmacy, Cluj-Napoca |
| record_format | Article |
| series | Applied Medical Informatics |
| spelling | doaj-art-f1e7b0166a5c426b98ba5ee3607454fb2025-01-05T21:07:44ZengIuliu Hatieganu University of Medicine and Pharmacy, Cluj-NapocaApplied Medical Informatics2067-78552024-11-0146Suppl. 2Beyond the ROC Curve: Activity Monitoring to Evaluate Deep Learning Models in Clinical SettingsHyunwoo CHOO0Kyung Hyun LEE1Sungsoo HONG2Sungjun HONG3Ki-Byung LEE4Chang Youl LEE5AITRICS Inc., 218 Teheran-ro, Gangnam-gu, 06221Seoul, Republic of KoreaAITRICS Inc., 218 Teheran-ro, Gangnam-gu, 06221Seoul, Republic of KoreaAITRICS Inc., 218 Teheran-ro, Gangnam-gu, 06221Seoul, Republic of KoreaMedical AI Research Center, Research Institute for Future Medicine, Samsung Medical Center, 81 Irwon-ro, Gangnam-gu, 06351 Seoul, Republic of KoreaDivision of Pulmonary, Allergy and Critical Care Medicine, Hallym University Chuncheon Sacred Heart Hospital, 77 Sakju-ro, 24253 Chuncheon, Republic of KoreaDivision of Pulmonary, Allergy and Critical Care Medicine, Hallym University Chuncheon Sacred Heart Hospital, 77 Sakju-ro, 24253 Chuncheon, Republic of Korea We evaluated ‘VITALCARE-SEPS’, a deep learning model for sepsis prediction, using the activity monitoring operator characteristics curve with two different scoring algorithms. This evaluation is crucial as the AMOC curve addresses the time-dependent nature of predictions, providing a more nuanced performance assessment than traditional ROC metrics. Our findings demonstrate that the AMOC curve significantly enhances the evaluation of time-series predictions, enabling more accurate and continuous performance monitoring of machine learning models in clinical settings. This approach can improve model deployment and ultimately lead to better patient outcomes in healthcare. https://ami.info.umfcluj.ro/index.php/AMI/article/view/1074Deep learningSepsisActivity monitoringROC (receiver operating characteristic) curve |
| spellingShingle | Hyunwoo CHOO Kyung Hyun LEE Sungsoo HONG Sungjun HONG Ki-Byung LEE Chang Youl LEE Beyond the ROC Curve: Activity Monitoring to Evaluate Deep Learning Models in Clinical Settings Applied Medical Informatics Deep learning Sepsis Activity monitoring ROC (receiver operating characteristic) curve |
| title | Beyond the ROC Curve: Activity Monitoring to Evaluate Deep Learning Models in Clinical Settings |
| title_full | Beyond the ROC Curve: Activity Monitoring to Evaluate Deep Learning Models in Clinical Settings |
| title_fullStr | Beyond the ROC Curve: Activity Monitoring to Evaluate Deep Learning Models in Clinical Settings |
| title_full_unstemmed | Beyond the ROC Curve: Activity Monitoring to Evaluate Deep Learning Models in Clinical Settings |
| title_short | Beyond the ROC Curve: Activity Monitoring to Evaluate Deep Learning Models in Clinical Settings |
| title_sort | beyond the roc curve activity monitoring to evaluate deep learning models in clinical settings |
| topic | Deep learning Sepsis Activity monitoring ROC (receiver operating characteristic) curve |
| url | https://ami.info.umfcluj.ro/index.php/AMI/article/view/1074 |
| work_keys_str_mv | AT hyunwoochoo beyondtheroccurveactivitymonitoringtoevaluatedeeplearningmodelsinclinicalsettings AT kyunghyunlee beyondtheroccurveactivitymonitoringtoevaluatedeeplearningmodelsinclinicalsettings AT sungsoohong beyondtheroccurveactivitymonitoringtoevaluatedeeplearningmodelsinclinicalsettings AT sungjunhong beyondtheroccurveactivitymonitoringtoevaluatedeeplearningmodelsinclinicalsettings AT kibyunglee beyondtheroccurveactivitymonitoringtoevaluatedeeplearningmodelsinclinicalsettings AT changyoullee beyondtheroccurveactivitymonitoringtoevaluatedeeplearningmodelsinclinicalsettings |