An interpretable framework for inter-observer agreement measurements in TILs scoring on histopathological breast images: A proof-of-principle study.
Breast cancer, a widespread and life-threatening disease, necessitates precise diagnostic tools for improved patient outcomes. Tumor-Infiltrating Lymphocytes (TILs), reflective of the immune response against cancer cells, are pivotal in understanding breast cancer behavior. However, inter-observer v...
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2024-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://doi.org/10.1371/journal.pone.0314450 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841555413030404096 |
|---|---|
| author | Abdulkerim Capar Dursun Ali Ekinci Mucahit Ertano M Khalid Khan Niazi Erva Bengu Balaban Ibrahim Aloglu Meryem Dogan Ziyu Su Fugen Vardar Aker Metin Nafi Gurcan |
| author_facet | Abdulkerim Capar Dursun Ali Ekinci Mucahit Ertano M Khalid Khan Niazi Erva Bengu Balaban Ibrahim Aloglu Meryem Dogan Ziyu Su Fugen Vardar Aker Metin Nafi Gurcan |
| author_sort | Abdulkerim Capar |
| collection | DOAJ |
| description | Breast cancer, a widespread and life-threatening disease, necessitates precise diagnostic tools for improved patient outcomes. Tumor-Infiltrating Lymphocytes (TILs), reflective of the immune response against cancer cells, are pivotal in understanding breast cancer behavior. However, inter-observer variability in TILs scoring methods poses challenges to reliable assessments. This study introduces a novel and interpretable proof-of-principle framework comprising two innovative inter-observer agreement measures. The first method, Boundary-Weighted Fleiss' Kappa (BWFK), addresses tissue segmentation predictions, focusing on mitigating disagreements along tissue boundaries. BWFK enhances the accuracy of stromal segmentation, providing a nuanced assessment of inter-observer agreement. The second proposed method, the Distance Based Cell Agreement Algorithm (DBCAA), eliminates the need for ground truth annotations in cell detection predictions. This innovative approach offers versatility across histopathological analyses, overcoming data availability challenges. Both methods were applied to assess inter-observer agreement using a clinical image dataset consisting of 25 images of invasive ductal breast carcinoma tissue, each annotated by four pathologists, serving as a proof-of-principle. Experimental investigations demonstrated that the BWFK method yielded gains of up to 32% compared to the standard Fleiss' Kappa model. Furthermore, a procedure for conducting clinical validations of artificial intelligence (AI) based cell detection methods was elucidated. Thoroughly validated on a clinical dataset, the framework contributes to standardized, reliable, and interpretable inter-observer agreement assessments. This study is the first examination of inter-observer agreements in stromal segmentation and lymphocyte detection for the TILs scoring problem. The study emphasizes the potential impact of these measures in advancing histopathological image analysis, fostering consensus in TILs scoring, and ultimately improving breast cancer diagnostics and treatment planning. The source code and implementation guide for this study are accessible on our GitHub page, and the full clinical dataset is available for academic and research purposes on Kaggle. |
| format | Article |
| id | doaj-art-c25c2e672dd74ed292cae46f6c37740b |
| institution | Kabale University |
| issn | 1932-6203 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj-art-c25c2e672dd74ed292cae46f6c37740b2025-01-08T05:33:29ZengPublic Library of Science (PLoS)PLoS ONE1932-62032024-01-011912e031445010.1371/journal.pone.0314450An interpretable framework for inter-observer agreement measurements in TILs scoring on histopathological breast images: A proof-of-principle study.Abdulkerim CaparDursun Ali EkinciMucahit ErtanoM Khalid Khan NiaziErva Bengu BalabanIbrahim AlogluMeryem DoganZiyu SuFugen Vardar AkerMetin Nafi GurcanBreast cancer, a widespread and life-threatening disease, necessitates precise diagnostic tools for improved patient outcomes. Tumor-Infiltrating Lymphocytes (TILs), reflective of the immune response against cancer cells, are pivotal in understanding breast cancer behavior. However, inter-observer variability in TILs scoring methods poses challenges to reliable assessments. This study introduces a novel and interpretable proof-of-principle framework comprising two innovative inter-observer agreement measures. The first method, Boundary-Weighted Fleiss' Kappa (BWFK), addresses tissue segmentation predictions, focusing on mitigating disagreements along tissue boundaries. BWFK enhances the accuracy of stromal segmentation, providing a nuanced assessment of inter-observer agreement. The second proposed method, the Distance Based Cell Agreement Algorithm (DBCAA), eliminates the need for ground truth annotations in cell detection predictions. This innovative approach offers versatility across histopathological analyses, overcoming data availability challenges. Both methods were applied to assess inter-observer agreement using a clinical image dataset consisting of 25 images of invasive ductal breast carcinoma tissue, each annotated by four pathologists, serving as a proof-of-principle. Experimental investigations demonstrated that the BWFK method yielded gains of up to 32% compared to the standard Fleiss' Kappa model. Furthermore, a procedure for conducting clinical validations of artificial intelligence (AI) based cell detection methods was elucidated. Thoroughly validated on a clinical dataset, the framework contributes to standardized, reliable, and interpretable inter-observer agreement assessments. This study is the first examination of inter-observer agreements in stromal segmentation and lymphocyte detection for the TILs scoring problem. The study emphasizes the potential impact of these measures in advancing histopathological image analysis, fostering consensus in TILs scoring, and ultimately improving breast cancer diagnostics and treatment planning. The source code and implementation guide for this study are accessible on our GitHub page, and the full clinical dataset is available for academic and research purposes on Kaggle.https://doi.org/10.1371/journal.pone.0314450 |
| spellingShingle | Abdulkerim Capar Dursun Ali Ekinci Mucahit Ertano M Khalid Khan Niazi Erva Bengu Balaban Ibrahim Aloglu Meryem Dogan Ziyu Su Fugen Vardar Aker Metin Nafi Gurcan An interpretable framework for inter-observer agreement measurements in TILs scoring on histopathological breast images: A proof-of-principle study. PLoS ONE |
| title | An interpretable framework for inter-observer agreement measurements in TILs scoring on histopathological breast images: A proof-of-principle study. |
| title_full | An interpretable framework for inter-observer agreement measurements in TILs scoring on histopathological breast images: A proof-of-principle study. |
| title_fullStr | An interpretable framework for inter-observer agreement measurements in TILs scoring on histopathological breast images: A proof-of-principle study. |
| title_full_unstemmed | An interpretable framework for inter-observer agreement measurements in TILs scoring on histopathological breast images: A proof-of-principle study. |
| title_short | An interpretable framework for inter-observer agreement measurements in TILs scoring on histopathological breast images: A proof-of-principle study. |
| title_sort | interpretable framework for inter observer agreement measurements in tils scoring on histopathological breast images a proof of principle study |
| url | https://doi.org/10.1371/journal.pone.0314450 |
| work_keys_str_mv | AT abdulkerimcapar aninterpretableframeworkforinterobserveragreementmeasurementsintilsscoringonhistopathologicalbreastimagesaproofofprinciplestudy AT dursunaliekinci aninterpretableframeworkforinterobserveragreementmeasurementsintilsscoringonhistopathologicalbreastimagesaproofofprinciplestudy AT mucahitertano aninterpretableframeworkforinterobserveragreementmeasurementsintilsscoringonhistopathologicalbreastimagesaproofofprinciplestudy AT mkhalidkhanniazi aninterpretableframeworkforinterobserveragreementmeasurementsintilsscoringonhistopathologicalbreastimagesaproofofprinciplestudy AT ervabengubalaban aninterpretableframeworkforinterobserveragreementmeasurementsintilsscoringonhistopathologicalbreastimagesaproofofprinciplestudy AT ibrahimaloglu aninterpretableframeworkforinterobserveragreementmeasurementsintilsscoringonhistopathologicalbreastimagesaproofofprinciplestudy AT meryemdogan aninterpretableframeworkforinterobserveragreementmeasurementsintilsscoringonhistopathologicalbreastimagesaproofofprinciplestudy AT ziyusu aninterpretableframeworkforinterobserveragreementmeasurementsintilsscoringonhistopathologicalbreastimagesaproofofprinciplestudy AT fugenvardaraker aninterpretableframeworkforinterobserveragreementmeasurementsintilsscoringonhistopathologicalbreastimagesaproofofprinciplestudy AT metinnafigurcan aninterpretableframeworkforinterobserveragreementmeasurementsintilsscoringonhistopathologicalbreastimagesaproofofprinciplestudy AT abdulkerimcapar interpretableframeworkforinterobserveragreementmeasurementsintilsscoringonhistopathologicalbreastimagesaproofofprinciplestudy AT dursunaliekinci interpretableframeworkforinterobserveragreementmeasurementsintilsscoringonhistopathologicalbreastimagesaproofofprinciplestudy AT mucahitertano interpretableframeworkforinterobserveragreementmeasurementsintilsscoringonhistopathologicalbreastimagesaproofofprinciplestudy AT mkhalidkhanniazi interpretableframeworkforinterobserveragreementmeasurementsintilsscoringonhistopathologicalbreastimagesaproofofprinciplestudy AT ervabengubalaban interpretableframeworkforinterobserveragreementmeasurementsintilsscoringonhistopathologicalbreastimagesaproofofprinciplestudy AT ibrahimaloglu interpretableframeworkforinterobserveragreementmeasurementsintilsscoringonhistopathologicalbreastimagesaproofofprinciplestudy AT meryemdogan interpretableframeworkforinterobserveragreementmeasurementsintilsscoringonhistopathologicalbreastimagesaproofofprinciplestudy AT ziyusu interpretableframeworkforinterobserveragreementmeasurementsintilsscoringonhistopathologicalbreastimagesaproofofprinciplestudy AT fugenvardaraker interpretableframeworkforinterobserveragreementmeasurementsintilsscoringonhistopathologicalbreastimagesaproofofprinciplestudy AT metinnafigurcan interpretableframeworkforinterobserveragreementmeasurementsintilsscoringonhistopathologicalbreastimagesaproofofprinciplestudy |