Recommended antibiotic treatment agreement between infectious diseases specialists and ChatGPT®
Abstract Background Antimicrobial resistance is a global threat to public health. Chat Generative Pre-trained Transformer (ChatGPT®) is a language model tool based on artificial intelligence. ChatGPT® could analyze data from antimicrobial susceptibility tests in real time, especially in places where...
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2025-01-01
|
| Series: | BMC Infectious Diseases |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12879-024-10426-9 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841545019297628160 |
|---|---|
| author | Santiago Montiel-Romero Sandra Rajme-López Carla Marina Román-Montes Alvaro López-Iñiguez Héctor Orlando Rivera-Villegas Eric Ochoa-Hein María Fernanda González-Lara Alfredo Ponce-de-León Karla María Tamez-Torres Bernardo Alfonso Martinez-Guerra |
| author_facet | Santiago Montiel-Romero Sandra Rajme-López Carla Marina Román-Montes Alvaro López-Iñiguez Héctor Orlando Rivera-Villegas Eric Ochoa-Hein María Fernanda González-Lara Alfredo Ponce-de-León Karla María Tamez-Torres Bernardo Alfonso Martinez-Guerra |
| author_sort | Santiago Montiel-Romero |
| collection | DOAJ |
| description | Abstract Background Antimicrobial resistance is a global threat to public health. Chat Generative Pre-trained Transformer (ChatGPT®) is a language model tool based on artificial intelligence. ChatGPT® could analyze data from antimicrobial susceptibility tests in real time, especially in places where infectious diseases (ID) specialists are not available. We aimed to evaluate the agreement between ChatGPT® and ID specialists regarding appropriate antibiotic prescription in simulated cases. Methods Using data from microbiological isolates recovered in our center, we fabricated 100 cases of patients with different infections. Each case included age, infectious syndrome, isolated organism and complete antibiogram. Considering a precise set of instructions, the cases were introduced into ChatGPT® and presented to five ID specialists. For each case, we asked, (1) “What is the most appropriate antibiotic that should be prescribed to the patient in the clinical case?” and (2) “According to the interpretation of the antibiogram, what is the most probable mechanism of resistance?”. We then calculated the agreement between ID specialists and ChatGPT®, as well as Cohen’s kappa coefficient. Results Regarding the recommended antibiotic prescription, agreement between ID specialists and ChatGPT® was observed in 51/100 cases. The calculated kappa coefficient was 0.48. Agreement on antimicrobial resistance mechanisms was observed in 42/100 cases. The calculated kappa coefficient was 0.39. In a subanalysis according to infectious syndromes and microorganisms, Agreement (range 25 – 80%) and kappa coefficients (range 0.21–0.79) varied. Conclusion We found poor agreement between ID specialists and ChatGPT® regarding the recommended antibiotic management in simulated clinical cases. |
| format | Article |
| id | doaj-art-feddd9429bb949009f04032a0461ef84 |
| institution | Kabale University |
| issn | 1471-2334 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Infectious Diseases |
| spelling | doaj-art-feddd9429bb949009f04032a0461ef842025-01-12T12:09:45ZengBMCBMC Infectious Diseases1471-23342025-01-012511610.1186/s12879-024-10426-9Recommended antibiotic treatment agreement between infectious diseases specialists and ChatGPT®Santiago Montiel-Romero0Sandra Rajme-López1Carla Marina Román-Montes2Alvaro López-Iñiguez3Héctor Orlando Rivera-Villegas4Eric Ochoa-Hein5María Fernanda González-Lara6Alfredo Ponce-de-León7Karla María Tamez-Torres8Bernardo Alfonso Martinez-Guerra9Department of Infectious Diseases, Instituto Nacional de Ciencias Médicas y Nutrición Salvador ZubiránDepartment of Infectious Diseases, Instituto Nacional de Ciencias Médicas y Nutrición Salvador ZubiránDepartment of Infectious Diseases, Instituto Nacional de Ciencias Médicas y Nutrición Salvador ZubiránDepartment of Infectious Diseases, Instituto Nacional de Ciencias Médicas y Nutrición Salvador ZubiránDepartment of Infectious Diseases, Instituto Nacional de Ciencias Médicas y Nutrición Salvador ZubiránDepartment of Infectious Diseases, Instituto Nacional de Ciencias Médicas y Nutrición Salvador ZubiránDepartment of Infectious Diseases, Instituto Nacional de Ciencias Médicas y Nutrición Salvador ZubiránDepartment of Infectious Diseases, Instituto Nacional de Ciencias Médicas y Nutrición Salvador ZubiránDepartment of Infectious Diseases, Instituto Nacional de Ciencias Médicas y Nutrición Salvador ZubiránDepartment of Infectious Diseases, Instituto Nacional de Ciencias Médicas y Nutrición Salvador ZubiránAbstract Background Antimicrobial resistance is a global threat to public health. Chat Generative Pre-trained Transformer (ChatGPT®) is a language model tool based on artificial intelligence. ChatGPT® could analyze data from antimicrobial susceptibility tests in real time, especially in places where infectious diseases (ID) specialists are not available. We aimed to evaluate the agreement between ChatGPT® and ID specialists regarding appropriate antibiotic prescription in simulated cases. Methods Using data from microbiological isolates recovered in our center, we fabricated 100 cases of patients with different infections. Each case included age, infectious syndrome, isolated organism and complete antibiogram. Considering a precise set of instructions, the cases were introduced into ChatGPT® and presented to five ID specialists. For each case, we asked, (1) “What is the most appropriate antibiotic that should be prescribed to the patient in the clinical case?” and (2) “According to the interpretation of the antibiogram, what is the most probable mechanism of resistance?”. We then calculated the agreement between ID specialists and ChatGPT®, as well as Cohen’s kappa coefficient. Results Regarding the recommended antibiotic prescription, agreement between ID specialists and ChatGPT® was observed in 51/100 cases. The calculated kappa coefficient was 0.48. Agreement on antimicrobial resistance mechanisms was observed in 42/100 cases. The calculated kappa coefficient was 0.39. In a subanalysis according to infectious syndromes and microorganisms, Agreement (range 25 – 80%) and kappa coefficients (range 0.21–0.79) varied. Conclusion We found poor agreement between ID specialists and ChatGPT® regarding the recommended antibiotic management in simulated clinical cases.https://doi.org/10.1186/s12879-024-10426-9Artificial intelligenceMachine learningInfectious diseasesAntimicrobial resistance |
| spellingShingle | Santiago Montiel-Romero Sandra Rajme-López Carla Marina Román-Montes Alvaro López-Iñiguez Héctor Orlando Rivera-Villegas Eric Ochoa-Hein María Fernanda González-Lara Alfredo Ponce-de-León Karla María Tamez-Torres Bernardo Alfonso Martinez-Guerra Recommended antibiotic treatment agreement between infectious diseases specialists and ChatGPT® BMC Infectious Diseases Artificial intelligence Machine learning Infectious diseases Antimicrobial resistance |
| title | Recommended antibiotic treatment agreement between infectious diseases specialists and ChatGPT® |
| title_full | Recommended antibiotic treatment agreement between infectious diseases specialists and ChatGPT® |
| title_fullStr | Recommended antibiotic treatment agreement between infectious diseases specialists and ChatGPT® |
| title_full_unstemmed | Recommended antibiotic treatment agreement between infectious diseases specialists and ChatGPT® |
| title_short | Recommended antibiotic treatment agreement between infectious diseases specialists and ChatGPT® |
| title_sort | recommended antibiotic treatment agreement between infectious diseases specialists and chatgpt r |
| topic | Artificial intelligence Machine learning Infectious diseases Antimicrobial resistance |
| url | https://doi.org/10.1186/s12879-024-10426-9 |
| work_keys_str_mv | AT santiagomontielromero recommendedantibiotictreatmentagreementbetweeninfectiousdiseasesspecialistsandchatgpt AT sandrarajmelopez recommendedantibiotictreatmentagreementbetweeninfectiousdiseasesspecialistsandchatgpt AT carlamarinaromanmontes recommendedantibiotictreatmentagreementbetweeninfectiousdiseasesspecialistsandchatgpt AT alvarolopeziniguez recommendedantibiotictreatmentagreementbetweeninfectiousdiseasesspecialistsandchatgpt AT hectororlandoriveravillegas recommendedantibiotictreatmentagreementbetweeninfectiousdiseasesspecialistsandchatgpt AT ericochoahein recommendedantibiotictreatmentagreementbetweeninfectiousdiseasesspecialistsandchatgpt AT mariafernandagonzalezlara recommendedantibiotictreatmentagreementbetweeninfectiousdiseasesspecialistsandchatgpt AT alfredoponcedeleon recommendedantibiotictreatmentagreementbetweeninfectiousdiseasesspecialistsandchatgpt AT karlamariatameztorres recommendedantibiotictreatmentagreementbetweeninfectiousdiseasesspecialistsandchatgpt AT bernardoalfonsomartinezguerra recommendedantibiotictreatmentagreementbetweeninfectiousdiseasesspecialistsandchatgpt |