Artificial Intelligence Chatbot as a Companion for Cancer Patients About Most Common Questions: Analysis of Readability and Quality
Aim: Advances on artificial intelligence (AI) have led to development of AI chatbots and more people are using AI chatbots to seek answers to their questions every day. We conducted this study to investigate the readability and quality of answers generated by large language model AI chatbots as comp...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Galenos Publishing House
2025-04-01
|
| Series: | Acta Haematologica Oncologica Turcica |
| Subjects: | |
| Online Access: | https://actaoncologicaturcica.com/articles/artificial-intelligence-chatbot-as-a-companion-for-cancer-patients-about-most-common-questions-analysis-of-readability-and-quality/doi/ahot.galenos.2024.2024-8-2 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849338172696166400 |
|---|---|
| author | Efe Cem Erdat Elif Berna Köksoy Güngör Utkan |
| author_facet | Efe Cem Erdat Elif Berna Köksoy Güngör Utkan |
| author_sort | Efe Cem Erdat |
| collection | DOAJ |
| description | Aim: Advances on artificial intelligence (AI) have led to development of AI chatbots and more people are using AI chatbots to seek answers to their questions every day. We conducted this study to investigate the readability and quality of answers generated by large language model AI chatbots as companions in answering questions for cancer patients.
Methods: After surveying 508 patients admitted to the outpatient clinic of Ankara University Faculty of Medicine, Department of Medical Oncology, we selected the most frequently asked questions about the four most common cancer types and general cancer knowledge. We asked these questions of ChatGPT (an AI chatbot from OpenAI) and calculated readability and quality scores, and the statistical difference between suggested and calculated reading scores. Means and the t-tests (one-way and/or paired) were used for statistical analysis.
Results: A total of 57 questions, including those about colorectal, breast, lung, prostate cancer, and general cancer questions, were selected for analysis. The mean Flesch Reading Ease Score for all questions was 48.18 [standard deviation (SD) ±11.65], which was significantly lower than the suggested reading score of 60 points (p<0.01). The mean score for graded readability scores was 13.21 (SD ±2.49), which was consistent with college-level readability and significantly higher than a suggested value of 6th graders (p<0.01). The mean DISCERN score of all questions was 51.98 (SD ±7.27) and the Global Quality Score was 3.91 (SD ±0.69). Breast cancer responses were easier to read on graded scales (p=0.02) and had higher quality (p=0.05).
Conclusion: ChatGPT may be a good companion for cancer patients despite its limitations, but it should be used carefully. |
| format | Article |
| id | doaj-art-ffc846ae9dfc4c76bddde611a2a322aa |
| institution | Kabale University |
| issn | 3061-9947 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | Galenos Publishing House |
| record_format | Article |
| series | Acta Haematologica Oncologica Turcica |
| spelling | doaj-art-ffc846ae9dfc4c76bddde611a2a322aa2025-08-20T03:44:28ZengGalenos Publishing HouseActa Haematologica Oncologica Turcica3061-99472025-04-01581212710.4274/ahot.galenos.2024.2024-8-2Artificial Intelligence Chatbot as a Companion for Cancer Patients About Most Common Questions: Analysis of Readability and QualityEfe Cem Erdat0https://orcid.org/0000-0002-1250-1297Elif Berna Köksoy1https://orcid.org/0000-0002-6590-4444Güngör Utkan2https://orcid.org/0000-0001-8445-6993Ankara University Faculty of Medicine Department of Medical Oncology, Ankara, TürkiyeAnkara University Faculty of Medicine Department of Medical Oncology, Ankara, TürkiyeAnkara University Faculty of Medicine Department of Medical Oncology, Ankara, TürkiyeAim: Advances on artificial intelligence (AI) have led to development of AI chatbots and more people are using AI chatbots to seek answers to their questions every day. We conducted this study to investigate the readability and quality of answers generated by large language model AI chatbots as companions in answering questions for cancer patients. Methods: After surveying 508 patients admitted to the outpatient clinic of Ankara University Faculty of Medicine, Department of Medical Oncology, we selected the most frequently asked questions about the four most common cancer types and general cancer knowledge. We asked these questions of ChatGPT (an AI chatbot from OpenAI) and calculated readability and quality scores, and the statistical difference between suggested and calculated reading scores. Means and the t-tests (one-way and/or paired) were used for statistical analysis. Results: A total of 57 questions, including those about colorectal, breast, lung, prostate cancer, and general cancer questions, were selected for analysis. The mean Flesch Reading Ease Score for all questions was 48.18 [standard deviation (SD) ±11.65], which was significantly lower than the suggested reading score of 60 points (p<0.01). The mean score for graded readability scores was 13.21 (SD ±2.49), which was consistent with college-level readability and significantly higher than a suggested value of 6th graders (p<0.01). The mean DISCERN score of all questions was 51.98 (SD ±7.27) and the Global Quality Score was 3.91 (SD ±0.69). Breast cancer responses were easier to read on graded scales (p=0.02) and had higher quality (p=0.05). Conclusion: ChatGPT may be a good companion for cancer patients despite its limitations, but it should be used carefully.https://actaoncologicaturcica.com/articles/artificial-intelligence-chatbot-as-a-companion-for-cancer-patients-about-most-common-questions-analysis-of-readability-and-quality/doi/ahot.galenos.2024.2024-8-2artificial intelligencechatbotscancer educationchatgpt |
| spellingShingle | Efe Cem Erdat Elif Berna Köksoy Güngör Utkan Artificial Intelligence Chatbot as a Companion for Cancer Patients About Most Common Questions: Analysis of Readability and Quality Acta Haematologica Oncologica Turcica artificial intelligence chatbots cancer education chatgpt |
| title | Artificial Intelligence Chatbot as a Companion for Cancer Patients About Most Common Questions: Analysis of Readability and Quality |
| title_full | Artificial Intelligence Chatbot as a Companion for Cancer Patients About Most Common Questions: Analysis of Readability and Quality |
| title_fullStr | Artificial Intelligence Chatbot as a Companion for Cancer Patients About Most Common Questions: Analysis of Readability and Quality |
| title_full_unstemmed | Artificial Intelligence Chatbot as a Companion for Cancer Patients About Most Common Questions: Analysis of Readability and Quality |
| title_short | Artificial Intelligence Chatbot as a Companion for Cancer Patients About Most Common Questions: Analysis of Readability and Quality |
| title_sort | artificial intelligence chatbot as a companion for cancer patients about most common questions analysis of readability and quality |
| topic | artificial intelligence chatbots cancer education chatgpt |
| url | https://actaoncologicaturcica.com/articles/artificial-intelligence-chatbot-as-a-companion-for-cancer-patients-about-most-common-questions-analysis-of-readability-and-quality/doi/ahot.galenos.2024.2024-8-2 |
| work_keys_str_mv | AT efecemerdat artificialintelligencechatbotasacompanionforcancerpatientsaboutmostcommonquestionsanalysisofreadabilityandquality AT elifbernakoksoy artificialintelligencechatbotasacompanionforcancerpatientsaboutmostcommonquestionsanalysisofreadabilityandquality AT gungorutkan artificialintelligencechatbotasacompanionforcancerpatientsaboutmostcommonquestionsanalysisofreadabilityandquality |