Deep learning for tubes and lines detection in critical illness: Generalizability and comparison with residents
Background: Artificial intelligence (AI) has been proven useful for the assessment of tubes and lines on chest radiographs of general patients. However, validation on intensive care unit (ICU) patients remains imperative. Methods: This retrospective case-control study evaluated the performance of de...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2024-12-01
|
| Series: | European Journal of Radiology Open |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2352047724000480 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846122146014691328 |
|---|---|
| author | Pootipong Wongveerasin Trongtum Tongdee Pairash Saiviroonporn |
| author_facet | Pootipong Wongveerasin Trongtum Tongdee Pairash Saiviroonporn |
| author_sort | Pootipong Wongveerasin |
| collection | DOAJ |
| description | Background: Artificial intelligence (AI) has been proven useful for the assessment of tubes and lines on chest radiographs of general patients. However, validation on intensive care unit (ICU) patients remains imperative. Methods: This retrospective case-control study evaluated the performance of deep learning (DL) models for tubes and lines classification on both an external public dataset and a local dataset comprising 303 films randomly sampled from the ICU database. The endotracheal tubes (ETTs), central venous catheters (CVCs), and nasogastric tubes (NGTs) were classified into “Normal,” “Abnormal,” or “Borderline” positions by DL models with and without rule-based modification. Their performance was evaluated using an experienced radiologist as the standard reference. Results: The algorithm showed decreased performance on the local ICU dataset, compared to that of the external dataset, decreasing from the Area Under the Curve of Receiver (AUC) of 0.967 (95 % CI 0.965–0.973) to the AUC of 0.70 (95 % CI 0.68–0.77). Significant improvement in the ETT classification task was observed after modifications were made to the model to allow the use of the spatial relationship between line tips and reference anatomy with the improvement of the AUC, increasing from 0.71 (95 % CI 0.70 – 0.75) to 0.86 (95 % CI 0.83 – 0.94) Conclusions: The externally trained model exhibited limited generalizability on the local ICU dataset. Therefore, evaluating the performance of externally trained AI before integrating it into critical care routine is crucial. Rule-based algorithm may be used in combination with DL to improve results. |
| format | Article |
| id | doaj-art-eadeca859e2a46668506a5afc21d4034 |
| institution | Kabale University |
| issn | 2352-0477 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Elsevier |
| record_format | Article |
| series | European Journal of Radiology Open |
| spelling | doaj-art-eadeca859e2a46668506a5afc21d40342024-12-15T06:15:46ZengElsevierEuropean Journal of Radiology Open2352-04772024-12-0113100593Deep learning for tubes and lines detection in critical illness: Generalizability and comparison with residentsPootipong Wongveerasin0Trongtum Tongdee1Pairash Saiviroonporn2Corresponding author.; Department of Radiology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, ThailandDepartment of Radiology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, ThailandDepartment of Radiology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, ThailandBackground: Artificial intelligence (AI) has been proven useful for the assessment of tubes and lines on chest radiographs of general patients. However, validation on intensive care unit (ICU) patients remains imperative. Methods: This retrospective case-control study evaluated the performance of deep learning (DL) models for tubes and lines classification on both an external public dataset and a local dataset comprising 303 films randomly sampled from the ICU database. The endotracheal tubes (ETTs), central venous catheters (CVCs), and nasogastric tubes (NGTs) were classified into “Normal,” “Abnormal,” or “Borderline” positions by DL models with and without rule-based modification. Their performance was evaluated using an experienced radiologist as the standard reference. Results: The algorithm showed decreased performance on the local ICU dataset, compared to that of the external dataset, decreasing from the Area Under the Curve of Receiver (AUC) of 0.967 (95 % CI 0.965–0.973) to the AUC of 0.70 (95 % CI 0.68–0.77). Significant improvement in the ETT classification task was observed after modifications were made to the model to allow the use of the spatial relationship between line tips and reference anatomy with the improvement of the AUC, increasing from 0.71 (95 % CI 0.70 – 0.75) to 0.86 (95 % CI 0.83 – 0.94) Conclusions: The externally trained model exhibited limited generalizability on the local ICU dataset. Therefore, evaluating the performance of externally trained AI before integrating it into critical care routine is crucial. Rule-based algorithm may be used in combination with DL to improve results.http://www.sciencedirect.com/science/article/pii/S2352047724000480Tubes and lineschest radiographartificial intelligencedeep learninggeneralizabilityresidents |
| spellingShingle | Pootipong Wongveerasin Trongtum Tongdee Pairash Saiviroonporn Deep learning for tubes and lines detection in critical illness: Generalizability and comparison with residents European Journal of Radiology Open Tubes and lines chest radiograph artificial intelligence deep learning generalizability residents |
| title | Deep learning for tubes and lines detection in critical illness: Generalizability and comparison with residents |
| title_full | Deep learning for tubes and lines detection in critical illness: Generalizability and comparison with residents |
| title_fullStr | Deep learning for tubes and lines detection in critical illness: Generalizability and comparison with residents |
| title_full_unstemmed | Deep learning for tubes and lines detection in critical illness: Generalizability and comparison with residents |
| title_short | Deep learning for tubes and lines detection in critical illness: Generalizability and comparison with residents |
| title_sort | deep learning for tubes and lines detection in critical illness generalizability and comparison with residents |
| topic | Tubes and lines chest radiograph artificial intelligence deep learning generalizability residents |
| url | http://www.sciencedirect.com/science/article/pii/S2352047724000480 |
| work_keys_str_mv | AT pootipongwongveerasin deeplearningfortubesandlinesdetectionincriticalillnessgeneralizabilityandcomparisonwithresidents AT trongtumtongdee deeplearningfortubesandlinesdetectionincriticalillnessgeneralizabilityandcomparisonwithresidents AT pairashsaiviroonporn deeplearningfortubesandlinesdetectionincriticalillnessgeneralizabilityandcomparisonwithresidents |