Multi-label text classification via secondary use of large clinical real-world data sets
Abstract Procedural coding presents a taxing challenge for clinicians. However, recent advances in natural language processing offer a promising avenue for developing applications that assist clinicians, thereby alleviating their administrative burdens. This study seeks to create an application capa...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-11-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-024-76424-8 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841559434953752576 |
|---|---|
| author | Sai Pavan Kumar Veeranki Akhila Abdulnazar Diether Kramer Markus Kreuzthaler David Benjamin Lumenta |
| author_facet | Sai Pavan Kumar Veeranki Akhila Abdulnazar Diether Kramer Markus Kreuzthaler David Benjamin Lumenta |
| author_sort | Sai Pavan Kumar Veeranki |
| collection | DOAJ |
| description | Abstract Procedural coding presents a taxing challenge for clinicians. However, recent advances in natural language processing offer a promising avenue for developing applications that assist clinicians, thereby alleviating their administrative burdens. This study seeks to create an application capable of predicting procedure codes by analysing clinicians’ operative notes, aiming to streamline their workflow and enhance efficiency. We downstreamed an existing and a native German medical BERT model in a secondary use scenario, utilizing already coded surgery notes to model the coding procedure as a multi-label classification task. In comparison to the transformer-based architecture, we were levering the non-contextual model fastText, a convolutional neural network, a support vector machine and logistic regression for a comparative analysis of possible coding performance. About 350,000 notes were used for model adaption. By considering the top five suggested procedure codes from medBERT.de, surgeryBERT.at, fastText, a convolutional neural network, a support vector machine and a logistic regression, the mean average precision achieved was 0.880, 0.867, 0.870, 0.851, 0.870 and 0.805 respectively. Support vector machines performed better for surgery reports with a sequence length greater than 512, achieving a mean average precision of 0.872 in comparison to 0.840 for fastText, 0.837 for medBERT.de and 0.820 for surgeryBERT.at. A prototypical front-end application for coding support was additionally implemented. The problem of predicting procedure codes from a given operative report can be successfully modelled as a multi-label classification task, with a promising performance. Support vector machines as a classical machine learning method outperformed the non-contextual fastText approach. FastText with less demanding hardware resources has reached a similar performance to BERT-based models and has shown to be more suitable for explaining the predictions efficiently. |
| format | Article |
| id | doaj-art-668de0a041484cdda782a7af12f9ed62 |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-668de0a041484cdda782a7af12f9ed622025-01-05T12:29:33ZengNature PortfolioScientific Reports2045-23222024-11-0114111210.1038/s41598-024-76424-8Multi-label text classification via secondary use of large clinical real-world data setsSai Pavan Kumar Veeranki0Akhila Abdulnazar1Diether Kramer2Markus Kreuzthaler3David Benjamin Lumenta4Steiermärkische Krankenanstaltengesellschaft m.b.H. (KAGes)Institute for Medical Informatics, Statistics and Documentation, Medical University of GrazSteiermärkische Krankenanstaltengesellschaft m.b.H. (KAGes)Institute for Medical Informatics, Statistics and Documentation, Medical University of GrazResearch Unit for Digital Surgery, Division of Plastic, Aesthetic and Reconstructive Surgery, Department of Surgery, Medical University of GrazAbstract Procedural coding presents a taxing challenge for clinicians. However, recent advances in natural language processing offer a promising avenue for developing applications that assist clinicians, thereby alleviating their administrative burdens. This study seeks to create an application capable of predicting procedure codes by analysing clinicians’ operative notes, aiming to streamline their workflow and enhance efficiency. We downstreamed an existing and a native German medical BERT model in a secondary use scenario, utilizing already coded surgery notes to model the coding procedure as a multi-label classification task. In comparison to the transformer-based architecture, we were levering the non-contextual model fastText, a convolutional neural network, a support vector machine and logistic regression for a comparative analysis of possible coding performance. About 350,000 notes were used for model adaption. By considering the top five suggested procedure codes from medBERT.de, surgeryBERT.at, fastText, a convolutional neural network, a support vector machine and a logistic regression, the mean average precision achieved was 0.880, 0.867, 0.870, 0.851, 0.870 and 0.805 respectively. Support vector machines performed better for surgery reports with a sequence length greater than 512, achieving a mean average precision of 0.872 in comparison to 0.840 for fastText, 0.837 for medBERT.de and 0.820 for surgeryBERT.at. A prototypical front-end application for coding support was additionally implemented. The problem of predicting procedure codes from a given operative report can be successfully modelled as a multi-label classification task, with a promising performance. Support vector machines as a classical machine learning method outperformed the non-contextual fastText approach. FastText with less demanding hardware resources has reached a similar performance to BERT-based models and has shown to be more suitable for explaining the predictions efficiently.https://doi.org/10.1038/s41598-024-76424-8Natural language processingText classificationClinical real world dataSecondary use |
| spellingShingle | Sai Pavan Kumar Veeranki Akhila Abdulnazar Diether Kramer Markus Kreuzthaler David Benjamin Lumenta Multi-label text classification via secondary use of large clinical real-world data sets Scientific Reports Natural language processing Text classification Clinical real world data Secondary use |
| title | Multi-label text classification via secondary use of large clinical real-world data sets |
| title_full | Multi-label text classification via secondary use of large clinical real-world data sets |
| title_fullStr | Multi-label text classification via secondary use of large clinical real-world data sets |
| title_full_unstemmed | Multi-label text classification via secondary use of large clinical real-world data sets |
| title_short | Multi-label text classification via secondary use of large clinical real-world data sets |
| title_sort | multi label text classification via secondary use of large clinical real world data sets |
| topic | Natural language processing Text classification Clinical real world data Secondary use |
| url | https://doi.org/10.1038/s41598-024-76424-8 |
| work_keys_str_mv | AT saipavankumarveeranki multilabeltextclassificationviasecondaryuseoflargeclinicalrealworlddatasets AT akhilaabdulnazar multilabeltextclassificationviasecondaryuseoflargeclinicalrealworlddatasets AT dietherkramer multilabeltextclassificationviasecondaryuseoflargeclinicalrealworlddatasets AT markuskreuzthaler multilabeltextclassificationviasecondaryuseoflargeclinicalrealworlddatasets AT davidbenjaminlumenta multilabeltextclassificationviasecondaryuseoflargeclinicalrealworlddatasets |