Evaluating the pathological and clinical implications of errors made by an artificial intelligence colon biopsy screening tool
Objective Artificial intelligence (AI) tools for histological diagnosis offer great potential to healthcare, yet failure to understand their clinical context is delaying adoption. IGUANA (Interpretable Gland-Graphs using a Neural Aggregator) is an AI algorithm that can effectively classify colonic b...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2025-01-01
|
| Series: | BMJ Open Gastroenterology |
| Online Access: | https://bmjopengastro.bmj.com/content/12/1/e001649.full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841556927151079424 |
|---|---|
| author | Simon Graham Andrew Robinson Shivam Bhanderi Harriet Evans David Snead Naveen Sivakumar Abhilasha Patel |
| author_facet | Simon Graham Andrew Robinson Shivam Bhanderi Harriet Evans David Snead Naveen Sivakumar Abhilasha Patel |
| author_sort | Simon Graham |
| collection | DOAJ |
| description | Objective Artificial intelligence (AI) tools for histological diagnosis offer great potential to healthcare, yet failure to understand their clinical context is delaying adoption. IGUANA (Interpretable Gland-Graphs using a Neural Aggregator) is an AI algorithm that can effectively classify colonic biopsies into normal versus abnormal categories, designed to automatically report normal cases. We performed a retrospective pathological and clinical review of the errors made by IGUANA.Methods False negative (FN) errors were the primary focus due to the greatest propensity for harm. Pathological evaluation involved assessment of whole slide image (WSI) quality, precise diagnoses for each missed entity and identification of factors impeding diagnosis. Clinical evaluation scored the impact of each error on the patient and detailed the type of impact in terms of missed diagnosis, investigations or treatment.Results Across 5054 WSIs from 2080 UK National Health Service patients there were 220 FN errors across 164 cases (4.4% of WSI, 7.9% of cases). Diagnostic errors varied from cases of adenocarcinoma to mild inflammation. 88.4% of FN errors would have no impact on patient care, with only one error causing major patient harm. Factors that protected against harm included biopsies being low-risk polyps or diagnostic features were detected in other biopsies.Conclusion Most FN errors would not result in patient harm, suggesting that even with a 7.9% case-level error rate, this AI tool might be more suitable for adoption than statistics portray. Consideration of the clinical context of AI tool errors is essential to facilitate safe implementation. |
| format | Article |
| id | doaj-art-39679f36707a498293a0077f3fa1f393 |
| institution | Kabale University |
| issn | 2054-4774 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Open Gastroenterology |
| spelling | doaj-art-39679f36707a498293a0077f3fa1f3932025-01-07T05:00:08ZengBMJ Publishing GroupBMJ Open Gastroenterology2054-47742025-01-0112110.1136/bmjgast-2024-001649Evaluating the pathological and clinical implications of errors made by an artificial intelligence colon biopsy screening toolSimon Graham0Andrew Robinson1Shivam Bhanderi2Harriet Evans3David Snead4Naveen Sivakumar5Abhilasha Patel6Histofy, Coventry, UKHistopathology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UKDepartment of Colorectal and General Surgery, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UKUniversity of Warwick, Coventry, UKHistopathology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UKDepartment of Colorectal and General Surgery, George Eliot Hospital NHS Trust, Nuneaton, England, UKDepartment of Colorectal and General Surgery, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UKObjective Artificial intelligence (AI) tools for histological diagnosis offer great potential to healthcare, yet failure to understand their clinical context is delaying adoption. IGUANA (Interpretable Gland-Graphs using a Neural Aggregator) is an AI algorithm that can effectively classify colonic biopsies into normal versus abnormal categories, designed to automatically report normal cases. We performed a retrospective pathological and clinical review of the errors made by IGUANA.Methods False negative (FN) errors were the primary focus due to the greatest propensity for harm. Pathological evaluation involved assessment of whole slide image (WSI) quality, precise diagnoses for each missed entity and identification of factors impeding diagnosis. Clinical evaluation scored the impact of each error on the patient and detailed the type of impact in terms of missed diagnosis, investigations or treatment.Results Across 5054 WSIs from 2080 UK National Health Service patients there were 220 FN errors across 164 cases (4.4% of WSI, 7.9% of cases). Diagnostic errors varied from cases of adenocarcinoma to mild inflammation. 88.4% of FN errors would have no impact on patient care, with only one error causing major patient harm. Factors that protected against harm included biopsies being low-risk polyps or diagnostic features were detected in other biopsies.Conclusion Most FN errors would not result in patient harm, suggesting that even with a 7.9% case-level error rate, this AI tool might be more suitable for adoption than statistics portray. Consideration of the clinical context of AI tool errors is essential to facilitate safe implementation.https://bmjopengastro.bmj.com/content/12/1/e001649.full |
| spellingShingle | Simon Graham Andrew Robinson Shivam Bhanderi Harriet Evans David Snead Naveen Sivakumar Abhilasha Patel Evaluating the pathological and clinical implications of errors made by an artificial intelligence colon biopsy screening tool BMJ Open Gastroenterology |
| title | Evaluating the pathological and clinical implications of errors made by an artificial intelligence colon biopsy screening tool |
| title_full | Evaluating the pathological and clinical implications of errors made by an artificial intelligence colon biopsy screening tool |
| title_fullStr | Evaluating the pathological and clinical implications of errors made by an artificial intelligence colon biopsy screening tool |
| title_full_unstemmed | Evaluating the pathological and clinical implications of errors made by an artificial intelligence colon biopsy screening tool |
| title_short | Evaluating the pathological and clinical implications of errors made by an artificial intelligence colon biopsy screening tool |
| title_sort | evaluating the pathological and clinical implications of errors made by an artificial intelligence colon biopsy screening tool |
| url | https://bmjopengastro.bmj.com/content/12/1/e001649.full |
| work_keys_str_mv | AT simongraham evaluatingthepathologicalandclinicalimplicationsoferrorsmadebyanartificialintelligencecolonbiopsyscreeningtool AT andrewrobinson evaluatingthepathologicalandclinicalimplicationsoferrorsmadebyanartificialintelligencecolonbiopsyscreeningtool AT shivambhanderi evaluatingthepathologicalandclinicalimplicationsoferrorsmadebyanartificialintelligencecolonbiopsyscreeningtool AT harrietevans evaluatingthepathologicalandclinicalimplicationsoferrorsmadebyanartificialintelligencecolonbiopsyscreeningtool AT davidsnead evaluatingthepathologicalandclinicalimplicationsoferrorsmadebyanartificialintelligencecolonbiopsyscreeningtool AT naveensivakumar evaluatingthepathologicalandclinicalimplicationsoferrorsmadebyanartificialintelligencecolonbiopsyscreeningtool AT abhilashapatel evaluatingthepathologicalandclinicalimplicationsoferrorsmadebyanartificialintelligencecolonbiopsyscreeningtool |