The importance of clinical experience in AI-assisted corneal diagnosis: verification using intentional AI misleading

Abstract We developed an AI system capable of automatically classifying anterior eye images as either normal or indicative of corneal diseases. This study aims to investigate the influence of AI’s misleading guidance on ophthalmologists’ responses. This cross-sectional study included 30 cases each o...

Full description

Saved in:
Bibliographic Details
Main Authors: Hiroki Maehara, Yuta Ueno, Takefumi Yamaguchi, Yoshiyuki Kitaguchi, Dai Miyazaki, Ryohei Nejima, Takenori Inomata, Naoko Kato, Tai-ichiro Chikama, Jun Ominato, Tatsuya Yunoki, Kinya Tsubota, Masahiro Oda, Manabu Suzutani, Tetsuju Sekiryu, Tetsuro Oshika
Format: Article
Language:English
Published: Nature Portfolio 2025-01-01
Series:Scientific Reports
Subjects:
Online Access:https://doi.org/10.1038/s41598-025-85827-0
Tags: Add Tag
No Tags, Be the first to tag this record!
_version_ 1841544679569489920
author Hiroki Maehara
Yuta Ueno
Takefumi Yamaguchi
Yoshiyuki Kitaguchi
Dai Miyazaki
Ryohei Nejima
Takenori Inomata
Naoko Kato
Tai-ichiro Chikama
Jun Ominato
Tatsuya Yunoki
Kinya Tsubota
Masahiro Oda
Manabu Suzutani
Tetsuju Sekiryu
Tetsuro Oshika
author_facet Hiroki Maehara
Yuta Ueno
Takefumi Yamaguchi
Yoshiyuki Kitaguchi
Dai Miyazaki
Ryohei Nejima
Takenori Inomata
Naoko Kato
Tai-ichiro Chikama
Jun Ominato
Tatsuya Yunoki
Kinya Tsubota
Masahiro Oda
Manabu Suzutani
Tetsuju Sekiryu
Tetsuro Oshika
author_sort Hiroki Maehara
collection DOAJ
description Abstract We developed an AI system capable of automatically classifying anterior eye images as either normal or indicative of corneal diseases. This study aims to investigate the influence of AI’s misleading guidance on ophthalmologists’ responses. This cross-sectional study included 30 cases each of infectious and immunological keratitis. Responses regarding the presence of infection were collected from 7 corneal specialists and 16 non-corneal-specialist ophthalmologists, first based on the images alone and then after presenting the AI’s classification results. The AI’s diagnoses were deliberately altered to present a correct classification in 70% of the cases and incorrect in 30%. The overall accuracy of the ophthalmologists did not significantly change after AI assistance was introduced [75.2 ± 8.1%, 75.9 ± 7.2%, respectively (P = 0.59)]. In cases where the AI presented incorrect diagnoses, the accuracy of corneal specialists before and after AI assistance was showing no significant change [60.3 ± 35.2% and 53.2 ± 30.9%, respectively (P = 0.11)]. In contrast, the accuracy for non-corneal specialists dropped significantly from 54.5 ± 27.8% to 31.6 ± 29.3% (P < 0.001), especially in cases where the AI presented incorrect options. Less experienced ophthalmologists were misled due to incorrect AI guidance, but corneal specialists were not. Even with the introduction of AI diagnostic support systems, the importance of ophthalmologist’s experience remains crucial.
format Article
id doaj-art-8a56c0b7a6f647c89337ee684bbf8f3b
institution Kabale University
issn 2045-2322
language English
publishDate 2025-01-01
publisher Nature Portfolio
record_format Article
series Scientific Reports
spelling doaj-art-8a56c0b7a6f647c89337ee684bbf8f3b2025-01-12T12:21:11ZengNature PortfolioScientific Reports2045-23222025-01-011511910.1038/s41598-025-85827-0The importance of clinical experience in AI-assisted corneal diagnosis: verification using intentional AI misleadingHiroki Maehara0Yuta Ueno1Takefumi Yamaguchi2Yoshiyuki Kitaguchi3Dai Miyazaki4Ryohei Nejima5Takenori Inomata6Naoko Kato7Tai-ichiro Chikama8Jun Ominato9Tatsuya Yunoki10Kinya Tsubota11Masahiro Oda12Manabu Suzutani13Tetsuju Sekiryu14Tetsuro Oshika15Department of Ophthalmology, Fukushima Medical University School of MedicineDepartment of Ophthalmology, Faculty of Medicine, University of TsukubaDepartment of Ophthalmology, Tokyo Dental College Ichikawa General HospitalDepartment of Ophthalmology, Osaka University Graduate School of MedicineDivision of Ophthalmology and Visual Science, Faculty of Medicine, Tottori UniversityDepartment of Ophthalmology, Miyata Eye HospitalDepartment of Ophthalmology, Juntendo University Graduate School of MedicineDepartment of Ophthalmology, Tsukazaki HospitalDivision of Ophthalmology and Visual Science, Graduate School of Biomedical and Health Sciences, Hiroshima UniversityDivision of Ophthalmology and Visual Science, Graduate School of Medical and Dental Sciences, Niigata UniversityDepartment of Ophthalmology, University of ToyamaDepartment of Ophthalmology, Tokyo Medical UniversityGraduate School of Informatics, Nagoya UniversityDepartment of Ophthalmology, Fukushima Medical University School of MedicineDepartment of Ophthalmology, Fukushima Medical University School of MedicineDepartment of Ophthalmology, Faculty of Medicine, University of TsukubaAbstract We developed an AI system capable of automatically classifying anterior eye images as either normal or indicative of corneal diseases. This study aims to investigate the influence of AI’s misleading guidance on ophthalmologists’ responses. This cross-sectional study included 30 cases each of infectious and immunological keratitis. Responses regarding the presence of infection were collected from 7 corneal specialists and 16 non-corneal-specialist ophthalmologists, first based on the images alone and then after presenting the AI’s classification results. The AI’s diagnoses were deliberately altered to present a correct classification in 70% of the cases and incorrect in 30%. The overall accuracy of the ophthalmologists did not significantly change after AI assistance was introduced [75.2 ± 8.1%, 75.9 ± 7.2%, respectively (P = 0.59)]. In cases where the AI presented incorrect diagnoses, the accuracy of corneal specialists before and after AI assistance was showing no significant change [60.3 ± 35.2% and 53.2 ± 30.9%, respectively (P = 0.11)]. In contrast, the accuracy for non-corneal specialists dropped significantly from 54.5 ± 27.8% to 31.6 ± 29.3% (P < 0.001), especially in cases where the AI presented incorrect options. Less experienced ophthalmologists were misled due to incorrect AI guidance, but corneal specialists were not. Even with the introduction of AI diagnostic support systems, the importance of ophthalmologist’s experience remains crucial.https://doi.org/10.1038/s41598-025-85827-0AIArtificial intelligenceOcular surfaceMisleading AI guidanceAI assist
spellingShingle Hiroki Maehara
Yuta Ueno
Takefumi Yamaguchi
Yoshiyuki Kitaguchi
Dai Miyazaki
Ryohei Nejima
Takenori Inomata
Naoko Kato
Tai-ichiro Chikama
Jun Ominato
Tatsuya Yunoki
Kinya Tsubota
Masahiro Oda
Manabu Suzutani
Tetsuju Sekiryu
Tetsuro Oshika
The importance of clinical experience in AI-assisted corneal diagnosis: verification using intentional AI misleading
Scientific Reports
AI
Artificial intelligence
Ocular surface
Misleading AI guidance
AI assist
title The importance of clinical experience in AI-assisted corneal diagnosis: verification using intentional AI misleading
title_full The importance of clinical experience in AI-assisted corneal diagnosis: verification using intentional AI misleading
title_fullStr The importance of clinical experience in AI-assisted corneal diagnosis: verification using intentional AI misleading
title_full_unstemmed The importance of clinical experience in AI-assisted corneal diagnosis: verification using intentional AI misleading
title_short The importance of clinical experience in AI-assisted corneal diagnosis: verification using intentional AI misleading
title_sort importance of clinical experience in ai assisted corneal diagnosis verification using intentional ai misleading
topic AI
Artificial intelligence
Ocular surface
Misleading AI guidance
AI assist
url https://doi.org/10.1038/s41598-025-85827-0
work_keys_str_mv AT hirokimaehara theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT yutaueno theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT takefumiyamaguchi theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT yoshiyukikitaguchi theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT daimiyazaki theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT ryoheinejima theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT takenoriinomata theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT naokokato theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT taiichirochikama theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT junominato theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT tatsuyayunoki theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT kinyatsubota theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT masahirooda theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT manabusuzutani theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT tetsujusekiryu theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT tetsurooshika theimportanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT hirokimaehara importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT yutaueno importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT takefumiyamaguchi importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT yoshiyukikitaguchi importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT daimiyazaki importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT ryoheinejima importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT takenoriinomata importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT naokokato importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT taiichirochikama importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT junominato importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT tatsuyayunoki importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT kinyatsubota importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT masahirooda importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT manabusuzutani importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT tetsujusekiryu importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading
AT tetsurooshika importanceofclinicalexperienceinaiassistedcornealdiagnosisverificationusingintentionalaimisleading