Large language models can outperform humans in social situational judgments
Abstract Large language models (LLM) have been a catalyst for the public interest in artificial intelligence (AI). These technologies perform some knowledge-based tasks better and faster than human beings. However, whether AIs can correctly assess social situations and devise socially appropriate be...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-11-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-024-79048-0 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846171891334643712 |
|---|---|
| author | Justin M. Mittelstädt Julia Maier Panja Goerke Frank Zinn Michael Hermes |
| author_facet | Justin M. Mittelstädt Julia Maier Panja Goerke Frank Zinn Michael Hermes |
| author_sort | Justin M. Mittelstädt |
| collection | DOAJ |
| description | Abstract Large language models (LLM) have been a catalyst for the public interest in artificial intelligence (AI). These technologies perform some knowledge-based tasks better and faster than human beings. However, whether AIs can correctly assess social situations and devise socially appropriate behavior, is still unclear. We conducted an established Situational Judgment Test (SJT) with five different chatbots and compared their results with responses of human participants (N = 276). Claude, Copilot and you.com’s smart assistant performed significantly better than humans in proposing suitable behaviors in social situations. Moreover, their effectiveness rating of different behavior options aligned well with expert ratings. These results indicate that LLMs are capable of producing adept social judgments. While this constitutes an important requirement for the use as virtual social assistants, challenges and risks are still associated with their wide-spread use in social contexts. |
| format | Article |
| id | doaj-art-7dc4e0ec09424376a76d85971f75a6e2 |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-7dc4e0ec09424376a76d85971f75a6e22024-11-10T12:26:26ZengNature PortfolioScientific Reports2045-23222024-11-0114111010.1038/s41598-024-79048-0Large language models can outperform humans in social situational judgmentsJustin M. Mittelstädt0Julia Maier1Panja Goerke2Frank Zinn3Michael Hermes4Department of Aviation and Space Psychology, German Aerospace Center, Institute of Aerospace MedicineDepartment of Aviation and Space Psychology, German Aerospace Center, Institute of Aerospace MedicineDepartment of Aviation and Space Psychology, German Aerospace Center, Institute of Aerospace MedicineDepartment of Aviation and Space Psychology, German Aerospace Center, Institute of Aerospace MedicineDepartment of Aviation and Space Psychology, German Aerospace Center, Institute of Aerospace MedicineAbstract Large language models (LLM) have been a catalyst for the public interest in artificial intelligence (AI). These technologies perform some knowledge-based tasks better and faster than human beings. However, whether AIs can correctly assess social situations and devise socially appropriate behavior, is still unclear. We conducted an established Situational Judgment Test (SJT) with five different chatbots and compared their results with responses of human participants (N = 276). Claude, Copilot and you.com’s smart assistant performed significantly better than humans in proposing suitable behaviors in social situations. Moreover, their effectiveness rating of different behavior options aligned well with expert ratings. These results indicate that LLMs are capable of producing adept social judgments. While this constitutes an important requirement for the use as virtual social assistants, challenges and risks are still associated with their wide-spread use in social contexts.https://doi.org/10.1038/s41598-024-79048-0 |
| spellingShingle | Justin M. Mittelstädt Julia Maier Panja Goerke Frank Zinn Michael Hermes Large language models can outperform humans in social situational judgments Scientific Reports |
| title | Large language models can outperform humans in social situational judgments |
| title_full | Large language models can outperform humans in social situational judgments |
| title_fullStr | Large language models can outperform humans in social situational judgments |
| title_full_unstemmed | Large language models can outperform humans in social situational judgments |
| title_short | Large language models can outperform humans in social situational judgments |
| title_sort | large language models can outperform humans in social situational judgments |
| url | https://doi.org/10.1038/s41598-024-79048-0 |
| work_keys_str_mv | AT justinmmittelstadt largelanguagemodelscanoutperformhumansinsocialsituationaljudgments AT juliamaier largelanguagemodelscanoutperformhumansinsocialsituationaljudgments AT panjagoerke largelanguagemodelscanoutperformhumansinsocialsituationaljudgments AT frankzinn largelanguagemodelscanoutperformhumansinsocialsituationaljudgments AT michaelhermes largelanguagemodelscanoutperformhumansinsocialsituationaljudgments |