Partial Attention in Global Context and Local Interaction for Addressing Noisy Labels and Weighted Redundancies on Medical Images
Recently, the application of deep neural networks to detect anomalies on medical images has been facing the appearance of noisy labels, including overlapping objects and similar classes. Therefore, this study aims to address this challenge by proposing a unique attention module that can assist deep...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-12-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/25/1/163 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841548941319995392 |
|---|---|
| author | Minh Tai Pham Nguyen Minh Khue Phan Tran Tadashi Nakano Thi Hong Tran Quoc Duy Nam Nguyen |
| author_facet | Minh Tai Pham Nguyen Minh Khue Phan Tran Tadashi Nakano Thi Hong Tran Quoc Duy Nam Nguyen |
| author_sort | Minh Tai Pham Nguyen |
| collection | DOAJ |
| description | Recently, the application of deep neural networks to detect anomalies on medical images has been facing the appearance of noisy labels, including overlapping objects and similar classes. Therefore, this study aims to address this challenge by proposing a unique attention module that can assist deep neural networks in focusing on important object features in noisy medical image conditions. This module integrates global context modeling to create long-range dependencies and local interactions to enable channel attention ability by using 1D convolution that not only performs well with noisy labels but also consumes significantly less resources without any dimensionality reduction. The module is then named Global Context and Local Interaction (GCLI). We have further experimented and proposed a partial attention strategy for the proposed GCLI module, aiming to efficiently reduce weighted redundancies. This strategy utilizes a subset of channels for GCLI to produce attention weights instead of considering every single channel. As a result, this strategy can greatly reduce the risk of introducing weighted redundancies caused by modeling global context. For classification, our proposed method is able to assist ResNet34 in achieving up to 82.5% accuracy on the Chaoyang test set, which is the highest figure among the other SOTA attention modules without using any processing filter to reduce the effect of noisy labels. For object detection, the GCLI is able to boost the capability of YOLOv8 up to 52.1% mAP50 on the GRAZPEDWRI-DX test set, demonstrating the highest performance among other attention modules and ranking second in the mAP50 metric on the VinDR-CXR test set. In terms of model complexity, our proposed GCLI module can consume fewer extra parameters up to 225 times and has inference speed faster than 30% compared to the other attention modules. |
| format | Article |
| id | doaj-art-addf7c79f85140249642b0c84c7888b1 |
| institution | Kabale University |
| issn | 1424-8220 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-addf7c79f85140249642b0c84c7888b12025-01-10T13:21:05ZengMDPI AGSensors1424-82202024-12-0125116310.3390/s25010163Partial Attention in Global Context and Local Interaction for Addressing Noisy Labels and Weighted Redundancies on Medical ImagesMinh Tai Pham Nguyen0Minh Khue Phan Tran1Tadashi Nakano2Thi Hong Tran3Quoc Duy Nam Nguyen4Faculty of Advanced Program, Ho Chi Minh City Open University, Ho Chi Minh City 700000, VietnamFaculty of Information Technology, Ho Chi Minh City Open University, Ho Chi Minh City 700000, VietnamDepartment of Core Informatics, Graduate School of Informatics, Osaka Metropolitan University, Osaka 558-8585, JapanDepartment of Core Informatics, Graduate School of Informatics, Osaka Metropolitan University, Osaka 558-8585, JapanDepartment of Core Informatics, Graduate School of Informatics, Osaka Metropolitan University, Osaka 558-8585, JapanRecently, the application of deep neural networks to detect anomalies on medical images has been facing the appearance of noisy labels, including overlapping objects and similar classes. Therefore, this study aims to address this challenge by proposing a unique attention module that can assist deep neural networks in focusing on important object features in noisy medical image conditions. This module integrates global context modeling to create long-range dependencies and local interactions to enable channel attention ability by using 1D convolution that not only performs well with noisy labels but also consumes significantly less resources without any dimensionality reduction. The module is then named Global Context and Local Interaction (GCLI). We have further experimented and proposed a partial attention strategy for the proposed GCLI module, aiming to efficiently reduce weighted redundancies. This strategy utilizes a subset of channels for GCLI to produce attention weights instead of considering every single channel. As a result, this strategy can greatly reduce the risk of introducing weighted redundancies caused by modeling global context. For classification, our proposed method is able to assist ResNet34 in achieving up to 82.5% accuracy on the Chaoyang test set, which is the highest figure among the other SOTA attention modules without using any processing filter to reduce the effect of noisy labels. For object detection, the GCLI is able to boost the capability of YOLOv8 up to 52.1% mAP50 on the GRAZPEDWRI-DX test set, demonstrating the highest performance among other attention modules and ranking second in the mAP50 metric on the VinDR-CXR test set. In terms of model complexity, our proposed GCLI module can consume fewer extra parameters up to 225 times and has inference speed faster than 30% compared to the other attention modules.https://www.mdpi.com/1424-8220/25/1/163attention mechanismdeep learningnoisy labelsweighted redundancies |
| spellingShingle | Minh Tai Pham Nguyen Minh Khue Phan Tran Tadashi Nakano Thi Hong Tran Quoc Duy Nam Nguyen Partial Attention in Global Context and Local Interaction for Addressing Noisy Labels and Weighted Redundancies on Medical Images Sensors attention mechanism deep learning noisy labels weighted redundancies |
| title | Partial Attention in Global Context and Local Interaction for Addressing Noisy Labels and Weighted Redundancies on Medical Images |
| title_full | Partial Attention in Global Context and Local Interaction for Addressing Noisy Labels and Weighted Redundancies on Medical Images |
| title_fullStr | Partial Attention in Global Context and Local Interaction for Addressing Noisy Labels and Weighted Redundancies on Medical Images |
| title_full_unstemmed | Partial Attention in Global Context and Local Interaction for Addressing Noisy Labels and Weighted Redundancies on Medical Images |
| title_short | Partial Attention in Global Context and Local Interaction for Addressing Noisy Labels and Weighted Redundancies on Medical Images |
| title_sort | partial attention in global context and local interaction for addressing noisy labels and weighted redundancies on medical images |
| topic | attention mechanism deep learning noisy labels weighted redundancies |
| url | https://www.mdpi.com/1424-8220/25/1/163 |
| work_keys_str_mv | AT minhtaiphamnguyen partialattentioninglobalcontextandlocalinteractionforaddressingnoisylabelsandweightedredundanciesonmedicalimages AT minhkhuephantran partialattentioninglobalcontextandlocalinteractionforaddressingnoisylabelsandweightedredundanciesonmedicalimages AT tadashinakano partialattentioninglobalcontextandlocalinteractionforaddressingnoisylabelsandweightedredundanciesonmedicalimages AT thihongtran partialattentioninglobalcontextandlocalinteractionforaddressingnoisylabelsandweightedredundanciesonmedicalimages AT quocduynamnguyen partialattentioninglobalcontextandlocalinteractionforaddressingnoisylabelsandweightedredundanciesonmedicalimages |