Advancing Social Intelligence in AI Agents: Technical Challenges and Open Questions (2024.emnlp-main)
Copied to clipboard
| Challenge: | Building socially-intelligent AI agents involves creating agents that can sense, perceive, reason about, learn from, and respond to affect, behavior, and cognition of other agents. |
| Approach: | They propose a set of technical challenges and open questions for researchers to advance Social-AI. |
| Outcome: | The proposed frameworks are based on the social intelligence competencies that evolved over thousands of years in Homo sapiens and are expected to be the foundations for the development of social-intelligent AI agents. |
Similar Papers
Social Intelligence in the Age of LLMs (2025.naacl-tutorial)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are a powerful tool for integrating human-like communication and context-aware interactions into artificial systems. |
| Approach: | They propose to introduce and overview different aspects of artificial social intelligence and their relationship with LLMs by introducing scientific methods for evaluating social intelligence in LLM. |
| Outcome: | This tutorial will introduce scientific methods for evaluating social intelligence in LLMs, highlighting the key challenges, and identifying promising research directions. |
Language Agents: Foundations, Prospects, and Risks (2024.emnlp-tutorials)
Copied to clipboard
| Challenge: | Language agents are autonomous agents that can follow language instructions to perform diverse tasks in real-world or simulated environments. |
| Approach: | They propose to provide a conceptual framework for language agents and a comprehensive discussion on key topics. |
| Outcome: | The proposed tutorial provides a conceptual framework of language agents and comprehensive discussion on important topic areas. |
Social Intelligence Data Infrastructure: Structuring the Present and Navigating the Future (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing work on social intelligence in NLP does not provide a coherent subfield for researchers to analyze and identify research gaps and future directions. |
| Approach: | They build a social AI taxonomy and a data library of 480 NLP datasets to analyze existing datasets and evaluate language models’ performance in different social intelligence aspects. |
| Outcome: | The proposed infrastructure analyzes existing dataset efforts and evaluates language models’ performance in different social intelligence aspects. |
Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs (2022.emnlp-main)
Copied to clipboard
| Challenge: | We show that one of today’s largest language models lacks this kind of social intelligence out-of-the-box, using two tasks: SocialIQa and ToMi. |
| Approach: | They propose to use social intelligence and Theory of Mind to examine whether modern large-scale language models lack this kind of social intelligence out-of-the-box. |
| Outcome: | The proposed model lacks social intelligence out-of-the-box, and has well-below human accuracies on SocialIQa and ToMi, respectively. |
AgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarios (2025.naacl-long)
Copied to clipboard
Xinyi Mou, Jingcong Liang, Jiayu Lin, Xinnong Zhang, Xiawei Liu, Shiyue Yang, Rong Ye, Lei Chen, Haoyu Kuang, Xuanjing Huang, Zhongyu Wei
| Challenge: | Large language models are increasingly employed to empower autonomous agents to simulate human behavior. |
| Approach: | They propose to evaluate LLM-driven agents through multi-turn interactions using a bottom-up approach to create diverse social scenarios constructed from extensive scripts. |
| Outcome: | The proposed model evaluates LLM-driven agents through multi-turn interactions emphasizing goal completion and implicit reasoning. |
Towards large language model-based personal agents in the enterprise: Current trends and open problems (2023.findings-emnlp)
Copied to clipboard
Vinod Muthusamy, Yara Rizk, Kiran Kate, Praveen Venkateswaran, Vatche Isahagian, Ashu Gulati, Parijat Dube
| Challenge: | Existing large language models (LLMs) are brittle to input changes and can produce inconsistent results for the same inputs. |
| Approach: | They propose to use large language models to reason about complex goals and orchestrate a set of pluggable tools or APIs to accomplish a goal. |
| Outcome: | The proposed use cases have many open problems in an exciting area of NLP research, such as trust and explainability, consistency and reproducibility, and the need for new metrics and benchmarks. |
Research Community Perspectives on “Intelligence” and Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Despite the widespread use of ‘artificial intelligence’ (AI) framing in NLP research, it is not clear what researchers mean by ”intelligence”. |
| Approach: | They propose to use the term "AI" to describe the perception of a system as intelligent, but note that it is not accepted by the majority of respondents. |
| Outcome: | The results suggest that the perception of the current NLP systems as 'intelligent' is a minority position (29%). |
Connecting Language and Vision to Actions (P18-5)
Copied to clipboard
| Challenge: | Recent advances in language and vision have made incredible progress in describing images and interacting with visual content in a physical or embodied environment. |
| Approach: | This tutorial will provide an overview of the growing number of multimodal tasks and datasets that combine textual and visual understanding. |
| Outcome: | This tutorial will review the state-of-the-art approaches to selected tasks such as image captioning, visual question answering and visual dialog. |
Towards Effective and Efficient Multi-Agent Language Model Systems: Foundations, Prospects, and Applications (2026.acl-tutorials)
Copied to clipboard
| Challenge: | Multi-agent systems powered by large language models still face challenges . tutorial focuses on three core components to build effective and efficient systems . |
| Approach: | This tutorial introduces recent advances in building effective and efficient multi-agent LLM systems . it focuses on three core components: model distillation, dynamic routing, memory- and compute efficient serving . |
| Outcome: | This tutorial introduces state-of-the-art techniques for building efficient and efficient multi-agent LLM systems . it covers coordination and communication among agents, crucial for collective performance . |
AI Agents for the Science of Science: A Survey of Tasks, Architectures, Evaluations, and Challenges (2026.findings-acl)
Copied to clipboard
| Challenge: | The Science of Science (SciSc) examines how scientific knowledge is produced, evaluated, and transformed by utilizing large-scale scholarly and bibliometric data. |
| Approach: | They propose a task-centered taxonomy for AI agents that model citations, collaborations, and community dynamics. |
| Outcome: | The proposed taxonomy distinguishes agents as simulations from tools for empirical analysis and scientific workflows. |