The importance of clinical experience in AI-assisted corneal diagnosis: verification using intentional AI misleading
Abstract We developed an AI system capable of automatically classifying anterior eye images as either normal or indicative of corneal diseases. This study aims to investigate the influence of AI’s misleading guidance on ophthalmologists’ responses. This cross-sectional study included 30 cases each o...
Saved in:
Main Authors: | , , , , , , , , , , , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
Nature Portfolio
2025-01-01
|
Series: | Scientific Reports |
Subjects: | |
Online Access: | https://doi.org/10.1038/s41598-025-85827-0 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
_version_ | 1841544679569489920 |
---|---|
author | Hiroki Maehara Yuta Ueno Takefumi Yamaguchi Yoshiyuki Kitaguchi Dai Miyazaki Ryohei Nejima Takenori Inomata Naoko Kato Tai-ichiro Chikama Jun Ominato Tatsuya Yunoki Kinya Tsubota Masahiro Oda Manabu Suzutani Tetsuju Sekiryu Tetsuro Oshika |
author_facet | Hiroki Maehara Yuta Ueno Takefumi Yamaguchi Yoshiyuki Kitaguchi Dai Miyazaki Ryohei Nejima Takenori Inomata Naoko Kato Tai-ichiro Chikama Jun Ominato Tatsuya Yunoki Kinya Tsubota Masahiro Oda Manabu Suzutani Tetsuju Sekiryu Tetsuro Oshika |
author_sort | Hiroki Maehara |
collection | DOAJ |
description | Abstract We developed an AI system capable of automatically classifying anterior eye images as either normal or indicative of corneal diseases. This study aims to investigate the influence of AI’s misleading guidance on ophthalmologists’ responses. This cross-sectional study included 30 cases each of infectious and immunological keratitis. Responses regarding the presence of infection were collected from 7 corneal specialists and 16 non-corneal-specialist ophthalmologists, first based on the images alone and then after presenting the AI’s classification results. The AI’s diagnoses were deliberately altered to present a correct classification in 70% of the cases and incorrect in 30%. The overall accuracy of the ophthalmologists did not significantly change after AI assistance was introduced [75.2 ± 8.1%, 75.9 ± 7.2%, respectively (P = 0.59)]. In cases where the AI presented incorrect diagnoses, the accuracy of corneal specialists before and after AI assistance was showing no significant change [60.3 ± 35.2% and 53.2 ± 30.9%, respectively (P = 0.11)]. In contrast, the accuracy for non-corneal specialists dropped significantly from 54.5 ± 27.8% to 31.6 ± 29.3% (P < 0.001), especially in cases where the AI presented incorrect options. Less experienced ophthalmologists were misled due to incorrect AI guidance, but corneal specialists were not. Even with the introduction of AI diagnostic support systems, the importance of ophthalmologist’s experience remains crucial. |
format | Article |
id | doaj-art-8a56c0b7a6f647c89337ee684bbf8f3b |
institution | Kabale University |
issn | 2045-2322 |
language | English |
publishDate | 2025-01-01 |
publisher | Nature Portfolio |
record_format | Article |
series | Scientific Reports |
spelling | doaj-art-8a56c0b7a6f647c89337ee684bbf8f3b2025-01-12T12:21:11ZengNature PortfolioScientific Reports2045-23222025-01-011511910.1038/s41598-025-85827-0The importance of clinical experience in AI-assisted corneal diagnosis: verification using intentional AI misleadingHiroki Maehara0Yuta Ueno1Takefumi Yamaguchi2Yoshiyuki Kitaguchi3Dai Miyazaki4Ryohei Nejima5Takenori Inomata6Naoko Kato7Tai-ichiro Chikama8Jun Ominato9Tatsuya Yunoki10Kinya Tsubota11Masahiro Oda12Manabu Suzutani13Tetsuju Sekiryu14Tetsuro Oshika15Department of Ophthalmology, Fukushima Medical University School of MedicineDepartment of Ophthalmology, Faculty of Medicine, University of TsukubaDepartment of Ophthalmology, Tokyo Dental College Ichikawa General HospitalDepartment of Ophthalmology, Osaka University Graduate School of MedicineDivision of Ophthalmology and Visual Science, Faculty of Medicine, Tottori UniversityDepartment of Ophthalmology, Miyata Eye HospitalDepartment of Ophthalmology, Juntendo University Graduate School of MedicineDepartment of Ophthalmology, Tsukazaki HospitalDivision of Ophthalmology and Visual Science, Graduate School of Biomedical and Health Sciences, Hiroshima UniversityDivision of Ophthalmology and Visual Science, Graduate School of Medical and Dental Sciences, Niigata UniversityDepartment of Ophthalmology, University of ToyamaDepartment of Ophthalmology, Tokyo Medical UniversityGraduate School of Informatics, Nagoya UniversityDepartment of Ophthalmology, Fukushima Medical University School of MedicineDepartment of Ophthalmology, Fukushima Medical University School of MedicineDepartment of Ophthalmology, Faculty of Medicine, University of TsukubaAbstract We developed an AI system capable of automatically classifying anterior eye images as either normal or indicative of corneal diseases. This study aims to investigate the influence of AI’s misleading guidance on ophthalmologists’ responses. This cross-sectional study included 30 cases each of infectious and immunological keratitis. Responses regarding the presence of infection were collected from 7 corneal specialists and 16 non-corneal-specialist ophthalmologists, first based on the images alone and then after presenting the AI’s classification results. The AI’s diagnoses were deliberately altered to present a correct classification in 70% of the cases and incorrect in 30%. The overall accuracy of the ophthalmologists did not significantly change after AI assistance was introduced [75.2 ± 8.1%, 75.9 ± 7.2%, respectively (P = 0.59)]. In cases where the AI presented incorrect diagnoses, the accuracy of corneal specialists before and after AI assistance was showing no significant change [60.3 ± 35.2% and 53.2 ± 30.9%, respectively (P = 0.11)]. In contrast, the accuracy for non-corneal specialists dropped significantly from 54.5 ± 27.8% to 31.6 ± 29.3% (P < 0.001), especially in cases where the AI presented incorrect options. Less experienced ophthalmologists were misled due to incorrect AI guidance, but corneal specialists were not. Even with the introduction of AI diagnostic support systems, the importance of ophthalmologist’s experience remains crucial.https://doi.org/10.1038/s41598-025-85827-0AIArtificial intelligenceOcular surfaceMisleading AI guidanceAI assist |
spellingShingle | Hiroki Maehara Yuta Ueno Takefumi Yamaguchi Yoshiyuki Kitaguchi Dai Miyazaki Ryohei Nejima Takenori Inomata Naoko Kato Tai-ichiro Chikama Jun Ominato Tatsuya Yunoki Kinya Tsubota Masahiro Oda Manabu Suzutani Tetsuju Sekiryu Tetsuro Oshika The importance of clinical experience in AI-assisted corneal diagnosis: verification using intentional AI misleading Scientific Reports AI Artificial intelligence Ocular surface Misleading AI guidance AI assist |
title | The importance of clinical experience in AI-assisted corneal diagnosis: verification using intentional AI misleading |
title_full | The importance of clinical experience in AI-assisted corneal diagnosis: verification using intentional AI misleading |
title_fullStr | The importance of clinical experience in AI-assisted corneal diagnosis: verification using intentional AI misleading |
title_full_unstemmed | The importance of clinical experience in AI-assisted corneal diagnosis: verification using intentional AI misleading |
title_short | The importance of clinical experience in AI-assisted corneal diagnosis: verification using intentional AI misleading |
title_sort | importance of clinical experience in ai assisted corneal diagnosis verification using intentional ai misleading |
topic | AI Artificial intelligence Ocular surface Misleading AI guidance AI assist |
url | https://doi.org/10.1038/s41598-025-85827-0 |
work_keys_str_mv | AT hirokimaehara theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT yutaueno theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT takefumiyamaguchi theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT yoshiyukikitaguchi theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT daimiyazaki theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT ryoheinejima theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT takenoriinomata theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT naokokato theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT taiichirochikama theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT junominato theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT tatsuyayunoki theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT kinyatsubota theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT masahirooda theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT manabusuzutani theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT tetsujusekiryu theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT tetsurooshika theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT hirokimaehara importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT yutaueno importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT takefumiyamaguchi importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT yoshiyukikitaguchi importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT daimiyazaki importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT ryoheinejima importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT takenoriinomata importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT naokokato importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT taiichirochikama importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT junominato importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT tatsuyayunoki importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT kinyatsubota importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT masahirooda importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT manabusuzutani importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT tetsujusekiryu importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading AT tetsurooshika importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading |