Evaluating the ability of large language models to emulate personality
Abstract For social sciences, recent advancements in Large Language Models (LLMs) have the potential to revolutionize the study of human behaviors by facilitating the creation of realistic agents characterized by a diverse range of individual differences. This research presents novel simulation stud...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-01-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-024-84109-5 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841559576926748672 |
|---|---|
| author | Yilei Wang Jiabao Zhao Deniz S. Ones Liang He Xin Xu |
| author_facet | Yilei Wang Jiabao Zhao Deniz S. Ones Liang He Xin Xu |
| author_sort | Yilei Wang |
| collection | DOAJ |
| description | Abstract For social sciences, recent advancements in Large Language Models (LLMs) have the potential to revolutionize the study of human behaviors by facilitating the creation of realistic agents characterized by a diverse range of individual differences. This research presents novel simulation studies assessing GPT-4’s ability to role-play real-world individuals with diverse big five personality profiles. In simulation 1, emulated personality responses exhibited superior internal consistency, but also a more distinct and structured factor organization compared to the human counterparts they were based on. Furthermore, these emulated scores exhibited remarkably high convergent validity with the human self-reported personality scale scores. Simulation 2 replicated these findings but demonstrated that the robustness of GPT-4’s role-playing appears to wane as the complexity of the roles increases. Introducing supplementary demographic information in conjunction with personality affected convergent validities for certain emulated traits. However, including additional demographic characteristics enhanced the validity of emulated personality scores for predicting external criteria. Collectively, the findings underscore a promising future of using LLMs to emulate realistic and real person-based agents with varied personality traits. The broader applied implications and avenues for future research are elaborated upon. |
| format | Article |
| id | doaj-art-ce6dc76e8bb84794a2e8fbec87b14c82 |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-ce6dc76e8bb84794a2e8fbec87b14c822025-01-05T12:22:44ZengNature PortfolioScientific Reports2045-23222025-01-011511910.1038/s41598-024-84109-5Evaluating the ability of large language models to emulate personalityYilei Wang0Jiabao Zhao1Deniz S. Ones2Liang He3Xin Xu4Shanghai Institute of AI for Education, East China Normal UniversitySchool of Computer Science and Technology, Donghua UniversityDepartment of Psychology, University of Minnesota at Twin CitiesShanghai Institute of AI for Education, East China Normal UniversityShanghai Institute of AI for Education, East China Normal UniversityAbstract For social sciences, recent advancements in Large Language Models (LLMs) have the potential to revolutionize the study of human behaviors by facilitating the creation of realistic agents characterized by a diverse range of individual differences. This research presents novel simulation studies assessing GPT-4’s ability to role-play real-world individuals with diverse big five personality profiles. In simulation 1, emulated personality responses exhibited superior internal consistency, but also a more distinct and structured factor organization compared to the human counterparts they were based on. Furthermore, these emulated scores exhibited remarkably high convergent validity with the human self-reported personality scale scores. Simulation 2 replicated these findings but demonstrated that the robustness of GPT-4’s role-playing appears to wane as the complexity of the roles increases. Introducing supplementary demographic information in conjunction with personality affected convergent validities for certain emulated traits. However, including additional demographic characteristics enhanced the validity of emulated personality scores for predicting external criteria. Collectively, the findings underscore a promising future of using LLMs to emulate realistic and real person-based agents with varied personality traits. The broader applied implications and avenues for future research are elaborated upon.https://doi.org/10.1038/s41598-024-84109-5 |
| spellingShingle | Yilei Wang Jiabao Zhao Deniz S. Ones Liang He Xin Xu Evaluating the ability of large language models to emulate personality Scientific Reports |
| title | Evaluating the ability of large language models to emulate personality |
| title_full | Evaluating the ability of large language models to emulate personality |
| title_fullStr | Evaluating the ability of large language models to emulate personality |
| title_full_unstemmed | Evaluating the ability of large language models to emulate personality |
| title_short | Evaluating the ability of large language models to emulate personality |
| title_sort | evaluating the ability of large language models to emulate personality |
| url | https://doi.org/10.1038/s41598-024-84109-5 |
| work_keys_str_mv | AT yileiwang evaluatingtheabilityoflargelanguagemodelstoemulatepersonality AT jiabaozhao evaluatingtheabilityoflargelanguagemodelstoemulatepersonality AT denizsones evaluatingtheabilityoflargelanguagemodelstoemulatepersonality AT lianghe evaluatingtheabilityoflargelanguagemodelstoemulatepersonality AT xinxu evaluatingtheabilityoflargelanguagemodelstoemulatepersonality |