The Comparative Emotional Capabilities of Five Popular Large Language Models
DOI:
https://doi.org/10.58445/rars.645Keywords:
LLM, artificial intelligence algorithms, emotional capabilitiesAbstract
Large language models (LLMs) are artificial intelligence algorithms which apply deep learning to large data sets to understand, summarize, generate and predict content. The most common application of LLMs are through generative AI specifically designed to generate text-based content. Starting in November of 2022, public awareness of LLMs has greatly increased with the release of ChatGPT3. While LLMs are powerful conveyors of information, this study explores their emotional capabilities. By examining 5 popular LLMs (BingAI, ChatGPT3.5, GoogleBard, ChatGPT4, HuggingChat) and their ability to comprehend, mimic, and convey emotion, this study attempts to answer which of these LLMs have the greatest ability to understand, mimic, and generate emotional content relating to joy, sadness, anger, fear, and disgust. Based on the results of the study’s survey, the most effective LLM with respect to emotional content was ChatGPT4, followed by ChatGPT3.5, HuggingChat, GoogleBard, and BingAI. A greater understanding of the comparative emotional capabilities of LLMs will be pivotal to assessing and predicting potential for therapeutic, medical, natural language processing, and personal use.
References
AlBadani, B., Shi, R., & Dong, J. (2022, January 14). A novel machine learning approach for sentiment analysis on Twitter incorporating the universal language model fine-tuning and SVM. MDPI. https://www.mdpi.com/2571-5577/5/1/13
Alharbi, A., Smith , P., & Lee, M. (2021, July 14). Enhancing contextualised language models with static character and word embeddings for emotional intensity and sentiment strength detection in Arabic tweets. Procedia Computer Science. https://www.sciencedirect.com/science/article/pii/S1877050921012084
Alloghani, M., Thron, C., & Subair, S. (1970, January 1). Cognitive computing, emotional intelligence, and artificial intelligence in Healthcare. SpringerLink. https://link.springer.com/chapter/10.1007/978-3-030-92245-0_5
Bridges, K. (1932). Emotional development in early infancy. Wiley. https://emotion.wisc.edu/wp-content/uploads/sites/1353/2021/03/Bridges-1932-Emotional-development-in-early-infancy.pdf
Churchill, K., & Lipman, L. (2016, March 8). Early childhood social and emotional development: Advancing the field of measurement. Journal of Applied Developmental Psychology. https://www.sciencedirect.com/science/article/pii/S0193397316300053?via%3Dihub
Cropanzano, R., Hareli, S., Johnson, N. A., Kelly, J. R., Kopelman, S., Mehu, M., Mussweiler, T., Newcombe, M. J., Parkinson, B., Sinaceur, M., Barsade, S. G., Bartel, C., Bos, N. D., Brown, W. M., Byron, K., Daft, R. L., DeSanctis, G., Ekman, P., Elfenbein, H. A., … Hatfield, E. (2011, July 2). Anger and happiness in virtual teams: Emotional influences of text and behavior on others’ affect in the absence of non-verbal cues. Organizational Behavior and Human Decision Processes. https://www.sciencedirect.com/science/article/abs/pii/S0749597811000756
Ekman, P. (1991). Universal Emotions. Psycology.
Frey, C. B., Eckel, C. C., Kabir, G., Steinhubl, S. R., Hinton, G., Darcy, A. M., Obermeyer, Z., Topol, E., Verghese, A., Naylor, C. D., & Cohen, I. G. (2019, November 18). Artificial Intelligence and the future of psychiatry: Insights from a global physician survey. Artificial Intelligence in Medicine. https://www.sciencedirect.com/science/article/abs/pii/S0933365719306505
Grose, J. (1970, January 1). Genuine versus deceptive emotional displays. SpringerLink. https://link.springer.com/chapter/10.1007/978-94-007-2404-4_8
Ho, M.-T. (2022, October 11). What is a Turing test for emotional AI? - ai & society. SpringerLink. https://link.springer.com/article/10.1007/s00146-022-01571-3
Holodynski, M. (1970, January 1). Milestones and mechanisms of emotional development. SpringerLink. https://link.springer.com/chapter/10.1007/978-0-387-09546-2_7
Jack, R., Garrod, O., & Schyns, P. (2014, January 2). Dynamic facial expressions of emotion transmit an evolving hierarchy of signals over time. Current Biology. https://www.sciencedirect.com/science/article/pii/S0960982213015194
Jo, E., Epstein, D. A., Jung, H., & Kim, Y.-H. (2023, April 1). Understanding the benefits and challenges of deploying conversational AI leveraging large language models for Public Health Intervention: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. ACM Conferences. https://dl.acm.org/doi/10.1145/3544548.3581503
Kollareth, D., Shirai, M., Helmy, M., & Russell, J. (2022). Deconstructing disgust as the emotion of violations of body and soul. APA PsycNet. https://psycnet.apa.org/record/2021-99197-001
Merriam, G. (2022). If A.I. Only had a heart: Why Artificial Intelligence Research Needs to ... If A.I. Only Had a Heart: Why Artificial Intelligence Research Needs to Take Emotions More Seriously. https://worldscientific.com/doi/abs/10.1142/S2705078521500120
Moshkin, V., Konstantinov, A., & Yarushkina, N. (2022, September 22). Application of the bert language model for sentiment analysis of social network posts. SpringerLink. https://link.springer.com/chapter/10.1007/978-3-030-59535-7_20
Nasiri, Y. (2023, April 18). A language-model-based chatbot that considers the user’s personality profile and emotions to support caregivers of people with dementia. BYU ScholarsArchive. https://scholarsarchive.byu.edu/etd/9862/
OpenAI. (2023, March 27).ChatGPT4 technical report. arXiv.org. https://arxiv.org/abs/2303.08774
Plutchik, R. (1960, May 6). The Multifactor-Analytic Theory of Emotion. Taylor & Francis Online. https://www.tandfonline.com/doi/abs/10.1080/00223980.1960.9916432
Prochazkova, E., & Kret, M. E. (2017, May 12). Connecting minds and sharing emotions through mimicry: A neurocognitive model of emotional contagion. Neuroscience & Biobehavioral Reviews. https://www.sciencedirect.com/science/article/pii/S0149763416306704
Russell, S. P., & Giner-Sorolla, R. (2011). Social justifications for moral emotions: When reasons for disgust are less elaborated than for anger. American Psychological Association. https://psycnet.apa.org/record/2011-08951-001
Russell, S. P., & Giner-Sorolla, R. (2013). Bodily moral disgust: What it is, how it is different from anger, and why it is an unreasoned emotion. APA Psych Net. https://psycnet.apa.org/record/2013-06811-004
Zheng, C., Sabour, S., Wen, J., Zhang, Z., & Huang, M. (2022, February). AugESC: Dialogue augmentation with large language models for emotional support conversation. NASA/ADS. https://ui.adsabs.harvard.edu/abs/2022arXiv220213047Z/abstract
Casas, J., Torche, S., Daher, K., Mugellini, E., & Khaled, O. A. (2021, January 7). Emotional paraphrasing using pre-trained language models | IEEE ... IEEE Xplore. https://ieeexplore.ieee.org/document/9666309
Goswamy, T., Singh, I., Barkati, A., & Modi, A. (2020, December). Adapting a language model for controlled affective text generation. ACL Anthology. https://aclanthology.org/2020.coling-main.251/
Heaton, C. T., & Schwartz, D. M. (2020, October 1). Language models as emotional classifiers for textual conversation: Proceedings of the 28th ACM International Conference on Multimedia. ACM Conferences. https://dl.acm.org/doi/abs/10.1145/3394171.3413755
Li, C., Wang, J., Zhang, Y., Zhu, K., Hou, W., Lian, J., Luo, F., Yang, Q., & Xie, X. (2023, October 20). Large language models understand and can be enhanced by emotional stimuli. arXiv.org. https://arxiv.org/abs/2307.11760
Patel, S. C., & Fan, J. (2023, January 1). Identification and description of emotions by current large language models. bioRxiv. https://www.biorxiv.org/content/10.1101/2023.07.17.549421v1.abstract
Rosenfeld, A. (2021, May). Better Metrics for evaluating explainable artificial intelligence. ResearchGate. https://www.researchgate.net/publication/349111351_Better_Metrics_for_Evaluating_Explainable_Artificial_Intelligence_Blue_Sky_Ideas_Track
Rosenfeld, A., & Richardson, A. (2019, May 13). Explainability in human–agent systems - autonomous agents and multi-agent systems. SpringerLink. https://link.springer.com/article/10.1007/s10458-019-09408-y
Santhanam, S., & Shaikh, S. (2019, November 25). Emotional neural language generation grounded in situational contexts. arXiv.org. https://arxiv.org/abs/1911.11161
Serapio-García, G., Safdari, M., Crepy, C., Sun, L., Fitz, S., Romero, P., Abdulhai, M., Faust, A., & Matarić, M. (2023, September 21). Personality traits in large language models. arXiv.org. https://arxiv.org/abs/2307.00184
Simmons, G. (2023, June 17). Moral mimicry: Large language models produce moral rationalizations tailored to political identity. arXiv.org. https://arxiv.org/abs/2209.12106
Wang, X., Li, X., Yin, Z., Wu, Y., & Jia, L. (2023, July 28). Emotional intelligence of large language models. arXiv.org. https://arxiv.org/abs/2307.09042
Downloads
Posted
Categories
License
Copyright (c) 2023 Nathan Klapach
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.