Use of Attention Maps to Enrich Discriminability in Deep Learning Prediction Models Using Longitudinal Data from Electronic Health Records
Background: In predictive modelling, particularly in fields such as healthcare, the importance of understanding the model’s behaviour rivals, if not surpasses, that of discriminability. To this end, attention mechanisms have been included in deep learning models for years. However, when comparing di...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-12-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/15/1/146 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841549429632401408 |
|---|---|
| author | Lucía A. Carrasco-Ribelles Margarita Cabrera-Bean Jose Llanes-Jurado Concepción Violán |
| author_facet | Lucía A. Carrasco-Ribelles Margarita Cabrera-Bean Jose Llanes-Jurado Concepción Violán |
| author_sort | Lucía A. Carrasco-Ribelles |
| collection | DOAJ |
| description | Background: In predictive modelling, particularly in fields such as healthcare, the importance of understanding the model’s behaviour rivals, if not surpasses, that of discriminability. To this end, attention mechanisms have been included in deep learning models for years. However, when comparing different models, the one with the best discriminability is usually chosen without considering the clinical plausibility of their predictions. Objective: In this work several attention-based deep learning architectures with increasing degrees of complexity were designed and compared aiming to study the balance between discriminability and plausibility with architecture complexity when working with longitudinal data from Electronic Health Records (EHRs). Methods: We developed four deep learning-based architectures with attention mechanisms that were progressively more complex to handle longitudinal data from EHRs. We evaluated their discriminability and resulting attention maps and compared them amongst architectures and different input processing approaches. We trained them on 10 years of data from EHRs from Catalonia (Spain) and evaluated them using a 5-fold cross-validation to predict 1-year all-cause mortality in a subsample of 500,000 people over 65 years of age. Results: Generally, the simplest architectures led to the best overall discriminability, slightly decreasing with complexity by up to 8.7%. However, the attention maps resulting from the simpler architectures were less informative and less clinically plausible compared to those from more complex architectures. Moreover, the latter could give attention weights both in the time and feature domains. Conclusions: Our results suggest that discriminability and more informative and clinically plausible attention maps do not always go together. Given the preferences within the healthcare field for enhanced explainability, establishing a balance with discriminability is imperative. |
| format | Article |
| id | doaj-art-2478f54b37c847068992b1a89d2755e3 |
| institution | Kabale University |
| issn | 2076-3417 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj-art-2478f54b37c847068992b1a89d2755e32025-01-10T13:14:36ZengMDPI AGApplied Sciences2076-34172024-12-0115114610.3390/app15010146Use of Attention Maps to Enrich Discriminability in Deep Learning Prediction Models Using Longitudinal Data from Electronic Health RecordsLucía A. Carrasco-Ribelles0Margarita Cabrera-Bean1Jose Llanes-Jurado2Concepción Violán3Institut Universitari d’Investigació en Atenció Primària Jordi Gol (IDIAP Jordi Gol), 08007 Barcelona, SpainDepartment of Signal Theory and Communications, Universitat Politècnica de Catalunya (UPC), 08034 Barcelona, SpainInstituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, Universitat Politècnica de València, 46022 Valencia, SpainUnitat de Suport a la Recerca Metropolitana Nord, Institut Universitari d’Investigació en Atenció Primària Jordi Gol (IDIAP Jordi Gol), 08303 Mataró, SpainBackground: In predictive modelling, particularly in fields such as healthcare, the importance of understanding the model’s behaviour rivals, if not surpasses, that of discriminability. To this end, attention mechanisms have been included in deep learning models for years. However, when comparing different models, the one with the best discriminability is usually chosen without considering the clinical plausibility of their predictions. Objective: In this work several attention-based deep learning architectures with increasing degrees of complexity were designed and compared aiming to study the balance between discriminability and plausibility with architecture complexity when working with longitudinal data from Electronic Health Records (EHRs). Methods: We developed four deep learning-based architectures with attention mechanisms that were progressively more complex to handle longitudinal data from EHRs. We evaluated their discriminability and resulting attention maps and compared them amongst architectures and different input processing approaches. We trained them on 10 years of data from EHRs from Catalonia (Spain) and evaluated them using a 5-fold cross-validation to predict 1-year all-cause mortality in a subsample of 500,000 people over 65 years of age. Results: Generally, the simplest architectures led to the best overall discriminability, slightly decreasing with complexity by up to 8.7%. However, the attention maps resulting from the simpler architectures were less informative and less clinically plausible compared to those from more complex architectures. Moreover, the latter could give attention weights both in the time and feature domains. Conclusions: Our results suggest that discriminability and more informative and clinically plausible attention maps do not always go together. Given the preferences within the healthcare field for enhanced explainability, establishing a balance with discriminability is imperative.https://www.mdpi.com/2076-3417/15/1/146attention mechanismclinical plausibilitydiscriminabilityelectronic health recordrecurrent neural networklongitudinal data |
| spellingShingle | Lucía A. Carrasco-Ribelles Margarita Cabrera-Bean Jose Llanes-Jurado Concepción Violán Use of Attention Maps to Enrich Discriminability in Deep Learning Prediction Models Using Longitudinal Data from Electronic Health Records Applied Sciences attention mechanism clinical plausibility discriminability electronic health record recurrent neural network longitudinal data |
| title | Use of Attention Maps to Enrich Discriminability in Deep Learning Prediction Models Using Longitudinal Data from Electronic Health Records |
| title_full | Use of Attention Maps to Enrich Discriminability in Deep Learning Prediction Models Using Longitudinal Data from Electronic Health Records |
| title_fullStr | Use of Attention Maps to Enrich Discriminability in Deep Learning Prediction Models Using Longitudinal Data from Electronic Health Records |
| title_full_unstemmed | Use of Attention Maps to Enrich Discriminability in Deep Learning Prediction Models Using Longitudinal Data from Electronic Health Records |
| title_short | Use of Attention Maps to Enrich Discriminability in Deep Learning Prediction Models Using Longitudinal Data from Electronic Health Records |
| title_sort | use of attention maps to enrich discriminability in deep learning prediction models using longitudinal data from electronic health records |
| topic | attention mechanism clinical plausibility discriminability electronic health record recurrent neural network longitudinal data |
| url | https://www.mdpi.com/2076-3417/15/1/146 |
| work_keys_str_mv | AT luciaacarrascoribelles useofattentionmapstoenrichdiscriminabilityindeeplearningpredictionmodelsusinglongitudinaldatafromelectronichealthrecords AT margaritacabrerabean useofattentionmapstoenrichdiscriminabilityindeeplearningpredictionmodelsusinglongitudinaldatafromelectronichealthrecords AT josellanesjurado useofattentionmapstoenrichdiscriminabilityindeeplearningpredictionmodelsusinglongitudinaldatafromelectronichealthrecords AT concepcionviolan useofattentionmapstoenrichdiscriminabilityindeeplearningpredictionmodelsusinglongitudinaldatafromelectronichealthrecords |