From manual clinical criteria to machine learning algorithms: Comparing outcome endpoints derived from diverse electronic health record data modalities.
<h4>Background</h4>Progression free survival (PFS) is a critical clinical outcome endpoint during cancer management and treatment evaluation. Yet, PFS is often missing from publicly available datasets due to the current subjective, expert, and time-intensive nature of generating PFS metr...
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2025-05-01
|
| Series: | PLOS Digital Health |
| Online Access: | https://doi.org/10.1371/journal.pdig.0000755 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849327795622117376 |
|---|---|
| author | Shreya Chappidi Mason J Belue Stephanie A Harmon Sarisha Jagasia Ying Zhuge Erdal Tasci Baris Turkbey Jatinder Singh Kevin Camphausen Andra V Krauze |
| author_facet | Shreya Chappidi Mason J Belue Stephanie A Harmon Sarisha Jagasia Ying Zhuge Erdal Tasci Baris Turkbey Jatinder Singh Kevin Camphausen Andra V Krauze |
| author_sort | Shreya Chappidi |
| collection | DOAJ |
| description | <h4>Background</h4>Progression free survival (PFS) is a critical clinical outcome endpoint during cancer management and treatment evaluation. Yet, PFS is often missing from publicly available datasets due to the current subjective, expert, and time-intensive nature of generating PFS metrics. Given emerging research in multi-modal machine learning (ML), we explored the benefits and challenges associated with mining different electronic health record (EHR) data modalities and automating extraction of PFS metrics via ML algorithms.<h4>Methods</h4>We analyzed EHR data from 92 pathology-proven GBM patients, obtaining 233 corticosteroid prescriptions, 2080 radiology reports, and 743 brain MRI scans. Three methods were developed to derive clinical PFS: 1) frequency analysis of corticosteroid prescriptions, 2) natural language processing (NLP) of reports, and 3) computer vision (CV) volumetric analysis of imaging. Outputs from these methods were compared to manually annotated clinical guideline PFS metrics.<h4>Results</h4>Employing data-driven methods, standalone progression rates were 63% (prescription), 78% (NLP), and 54% (CV), compared to the 99% progression rate from manually applied clinical guidelines using integrated data sources. The prescription method identified progression an average of 5.2 months later than the clinical standard, while the CV and NLP algorithms identified progression earlier by 2.6 and 6.9 months, respectively. While lesion growth is a clinical guideline progression indicator, only half of patients exhibited increasing contrast-enhancing tumor volumes during scan-based CV analysis.<h4>Conclusion</h4>Our results indicate that data-driven algorithms can extract tumor progression outcomes from existing EHR data. However, ML methods are subject to varying availability bias, supporting contextual information, and pre-processing resource burdens that influence the extracted PFS endpoint distributions. Our scan-based CV results also suggest that the automation of clinical criteria may not align with human intuition. Our findings indicate a need for improved data source integration, validation, and revisiting of clinical criteria in parallel to multi-modal ML algorithm development. |
| format | Article |
| id | doaj-art-fdd969b51a204107a87ff70ebe750b4c |
| institution | Kabale University |
| issn | 2767-3170 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLOS Digital Health |
| spelling | doaj-art-fdd969b51a204107a87ff70ebe750b4c2025-08-20T03:47:45ZengPublic Library of Science (PLoS)PLOS Digital Health2767-31702025-05-0145e000075510.1371/journal.pdig.0000755From manual clinical criteria to machine learning algorithms: Comparing outcome endpoints derived from diverse electronic health record data modalities.Shreya ChappidiMason J BelueStephanie A HarmonSarisha JagasiaYing ZhugeErdal TasciBaris TurkbeyJatinder SinghKevin CamphausenAndra V Krauze<h4>Background</h4>Progression free survival (PFS) is a critical clinical outcome endpoint during cancer management and treatment evaluation. Yet, PFS is often missing from publicly available datasets due to the current subjective, expert, and time-intensive nature of generating PFS metrics. Given emerging research in multi-modal machine learning (ML), we explored the benefits and challenges associated with mining different electronic health record (EHR) data modalities and automating extraction of PFS metrics via ML algorithms.<h4>Methods</h4>We analyzed EHR data from 92 pathology-proven GBM patients, obtaining 233 corticosteroid prescriptions, 2080 radiology reports, and 743 brain MRI scans. Three methods were developed to derive clinical PFS: 1) frequency analysis of corticosteroid prescriptions, 2) natural language processing (NLP) of reports, and 3) computer vision (CV) volumetric analysis of imaging. Outputs from these methods were compared to manually annotated clinical guideline PFS metrics.<h4>Results</h4>Employing data-driven methods, standalone progression rates were 63% (prescription), 78% (NLP), and 54% (CV), compared to the 99% progression rate from manually applied clinical guidelines using integrated data sources. The prescription method identified progression an average of 5.2 months later than the clinical standard, while the CV and NLP algorithms identified progression earlier by 2.6 and 6.9 months, respectively. While lesion growth is a clinical guideline progression indicator, only half of patients exhibited increasing contrast-enhancing tumor volumes during scan-based CV analysis.<h4>Conclusion</h4>Our results indicate that data-driven algorithms can extract tumor progression outcomes from existing EHR data. However, ML methods are subject to varying availability bias, supporting contextual information, and pre-processing resource burdens that influence the extracted PFS endpoint distributions. Our scan-based CV results also suggest that the automation of clinical criteria may not align with human intuition. Our findings indicate a need for improved data source integration, validation, and revisiting of clinical criteria in parallel to multi-modal ML algorithm development.https://doi.org/10.1371/journal.pdig.0000755 |
| spellingShingle | Shreya Chappidi Mason J Belue Stephanie A Harmon Sarisha Jagasia Ying Zhuge Erdal Tasci Baris Turkbey Jatinder Singh Kevin Camphausen Andra V Krauze From manual clinical criteria to machine learning algorithms: Comparing outcome endpoints derived from diverse electronic health record data modalities. PLOS Digital Health |
| title | From manual clinical criteria to machine learning algorithms: Comparing outcome endpoints derived from diverse electronic health record data modalities. |
| title_full | From manual clinical criteria to machine learning algorithms: Comparing outcome endpoints derived from diverse electronic health record data modalities. |
| title_fullStr | From manual clinical criteria to machine learning algorithms: Comparing outcome endpoints derived from diverse electronic health record data modalities. |
| title_full_unstemmed | From manual clinical criteria to machine learning algorithms: Comparing outcome endpoints derived from diverse electronic health record data modalities. |
| title_short | From manual clinical criteria to machine learning algorithms: Comparing outcome endpoints derived from diverse electronic health record data modalities. |
| title_sort | from manual clinical criteria to machine learning algorithms comparing outcome endpoints derived from diverse electronic health record data modalities |
| url | https://doi.org/10.1371/journal.pdig.0000755 |
| work_keys_str_mv | AT shreyachappidi frommanualclinicalcriteriatomachinelearningalgorithmscomparingoutcomeendpointsderivedfromdiverseelectronichealthrecorddatamodalities AT masonjbelue frommanualclinicalcriteriatomachinelearningalgorithmscomparingoutcomeendpointsderivedfromdiverseelectronichealthrecorddatamodalities AT stephanieaharmon frommanualclinicalcriteriatomachinelearningalgorithmscomparingoutcomeendpointsderivedfromdiverseelectronichealthrecorddatamodalities AT sarishajagasia frommanualclinicalcriteriatomachinelearningalgorithmscomparingoutcomeendpointsderivedfromdiverseelectronichealthrecorddatamodalities AT yingzhuge frommanualclinicalcriteriatomachinelearningalgorithmscomparingoutcomeendpointsderivedfromdiverseelectronichealthrecorddatamodalities AT erdaltasci frommanualclinicalcriteriatomachinelearningalgorithmscomparingoutcomeendpointsderivedfromdiverseelectronichealthrecorddatamodalities AT baristurkbey frommanualclinicalcriteriatomachinelearningalgorithmscomparingoutcomeendpointsderivedfromdiverseelectronichealthrecorddatamodalities AT jatindersingh frommanualclinicalcriteriatomachinelearningalgorithmscomparingoutcomeendpointsderivedfromdiverseelectronichealthrecorddatamodalities AT kevincamphausen frommanualclinicalcriteriatomachinelearningalgorithmscomparingoutcomeendpointsderivedfromdiverseelectronichealthrecorddatamodalities AT andravkrauze frommanualclinicalcriteriatomachinelearningalgorithmscomparingoutcomeendpointsderivedfromdiverseelectronichealthrecorddatamodalities |