Aligning knowledge concepts to whole slide images for precise histopathology image analysis
Abstract Due to the large size and lack of fine-grained annotation, Whole Slide Images (WSIs) analysis is commonly approached as a Multiple Instance Learning (MIL) problem. However, previous studies only learn from training data, posing a stark contrast to how human clinicians teach each other and r...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-12-01
|
| Series: | npj Digital Medicine |
| Online Access: | https://doi.org/10.1038/s41746-024-01411-2 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841559094000877568 |
|---|---|
| author | Weiqin Zhao Ziyu Guo Yinshuang Fan Yuming Jiang Maximus C. F. Yeung Lequan Yu |
| author_facet | Weiqin Zhao Ziyu Guo Yinshuang Fan Yuming Jiang Maximus C. F. Yeung Lequan Yu |
| author_sort | Weiqin Zhao |
| collection | DOAJ |
| description | Abstract Due to the large size and lack of fine-grained annotation, Whole Slide Images (WSIs) analysis is commonly approached as a Multiple Instance Learning (MIL) problem. However, previous studies only learn from training data, posing a stark contrast to how human clinicians teach each other and reason about histopathologic entities and factors. Here, we present a novel knowledge concept-based MIL framework, named ConcepPath, to fill this gap. Specifically, ConcepPath utilizes GPT-4 to induce reliable disease-specific human expert concepts from medical literature and incorporate them with a group of purely learnable concepts to extract complementary knowledge from training data. In ConcepPath, WSIs are aligned to these linguistic knowledge concepts by utilizing the pathology vision-language model as the basic building component. In the application of lung cancer subtyping, breast cancer HER2 scoring, and gastric cancer immunotherapy-sensitive subtyping tasks, ConcepPath significantly outperformed previous SOTA methods, which lacked the guidance of human expert knowledge. |
| format | Article |
| id | doaj-art-c3948fc85ca745dd897200af18562920 |
| institution | Kabale University |
| issn | 2398-6352 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | npj Digital Medicine |
| spelling | doaj-art-c3948fc85ca745dd897200af185629202025-01-05T12:47:28ZengNature Portfolionpj Digital Medicine2398-63522024-12-017111210.1038/s41746-024-01411-2Aligning knowledge concepts to whole slide images for precise histopathology image analysisWeiqin Zhao0Ziyu Guo1Yinshuang Fan2Yuming Jiang3Maximus C. F. Yeung4Lequan Yu5School of Computing and Data Science, The University of Hong KongSchool of Computing and Data Science, The University of Hong KongSchool of Computing and Data Science, The University of Hong KongSchool of Medicine, Wake Forest UniversityDepartment of Pathology, The University of Hong KongSchool of Computing and Data Science, The University of Hong KongAbstract Due to the large size and lack of fine-grained annotation, Whole Slide Images (WSIs) analysis is commonly approached as a Multiple Instance Learning (MIL) problem. However, previous studies only learn from training data, posing a stark contrast to how human clinicians teach each other and reason about histopathologic entities and factors. Here, we present a novel knowledge concept-based MIL framework, named ConcepPath, to fill this gap. Specifically, ConcepPath utilizes GPT-4 to induce reliable disease-specific human expert concepts from medical literature and incorporate them with a group of purely learnable concepts to extract complementary knowledge from training data. In ConcepPath, WSIs are aligned to these linguistic knowledge concepts by utilizing the pathology vision-language model as the basic building component. In the application of lung cancer subtyping, breast cancer HER2 scoring, and gastric cancer immunotherapy-sensitive subtyping tasks, ConcepPath significantly outperformed previous SOTA methods, which lacked the guidance of human expert knowledge.https://doi.org/10.1038/s41746-024-01411-2 |
| spellingShingle | Weiqin Zhao Ziyu Guo Yinshuang Fan Yuming Jiang Maximus C. F. Yeung Lequan Yu Aligning knowledge concepts to whole slide images for precise histopathology image analysis npj Digital Medicine |
| title | Aligning knowledge concepts to whole slide images for precise histopathology image analysis |
| title_full | Aligning knowledge concepts to whole slide images for precise histopathology image analysis |
| title_fullStr | Aligning knowledge concepts to whole slide images for precise histopathology image analysis |
| title_full_unstemmed | Aligning knowledge concepts to whole slide images for precise histopathology image analysis |
| title_short | Aligning knowledge concepts to whole slide images for precise histopathology image analysis |
| title_sort | aligning knowledge concepts to whole slide images for precise histopathology image analysis |
| url | https://doi.org/10.1038/s41746-024-01411-2 |
| work_keys_str_mv | AT weiqinzhao aligningknowledgeconceptstowholeslideimagesforprecisehistopathologyimageanalysis AT ziyuguo aligningknowledgeconceptstowholeslideimagesforprecisehistopathologyimageanalysis AT yinshuangfan aligningknowledgeconceptstowholeslideimagesforprecisehistopathologyimageanalysis AT yumingjiang aligningknowledgeconceptstowholeslideimagesforprecisehistopathologyimageanalysis AT maximuscfyeung aligningknowledgeconceptstowholeslideimagesforprecisehistopathologyimageanalysis AT lequanyu aligningknowledgeconceptstowholeslideimagesforprecisehistopathologyimageanalysis |