| Challenge: | a human-agent interaction corpus is a corpus of conversations between a user and an embodied conversational agent operated by a wizard of oz . data collected to create a 'corpus' with unexpected situations, such as misunderstandings, false information, and interruptions. |
| Approach: | They propose a public corpus for Human-Agent Interaction where the agent is controlled by a Wizard of Oz. |
| Outcome: | The proposed corpus is based on 15 conversations between users and a wizard of Oz agent . the data are used to create a corpus with unexpected situations, such as misunderstandings, false information, and interruptions. |
Similar Papers
The Niki and Julie Corpus: Collaborative Multimodal Dialogues between Humans, Robots, and Virtual Agents (L18-1)
Copied to clipboard
Ron Artstein, Jill Boberg, Alesia Gainer, Jonathan Gratch, Emmanuel Johnson, Anton Leuski, Gale Lucas, David Traum
| Challenge: | Niki and Julie corpus contains more than 600 dialogues between humans and robots . corpus includes audio and video recordings, results of ranking tasks, questionnaire responses . |
| Approach: | the corpus contains more than 600 dialogues between human participants and a robot . the dialogues are part of a collaborative item-ranking task designed to measure influence . |
| Outcome: | the corpus contains more than 600 dialogues between human participants and a robot or virtual agent . the dialogues contain conversational errors by the robot, which simulates typical of modern automated agents . |
LLM-Based Human-Agent Collaboration and Interaction Systems: A Survey (2026.findings-acl)
Copied to clipboard
Henry Peng Zou, Wei-Chieh Huang, Yaozu Wu, Jizhou Guo, Yankai Chen, Chunyu Miao, Hoang H Nguyen, Yue Zhou, Weizhi Zhang, Liancheng Fang, Hanrong Zhang, Fangxin Wang, Pengfei Zhang, Langzhou He, Yangning Li, Dongyuan Li, Renhe Jiang, Philip S. Yu
| Challenge: | Recent advances in large language models (LLMs) have sparked growing interest in building fully autonomous agents. |
| Approach: | They propose to integrate human-provided information, feedback, or control into the agent system to enhance system performance, reliability, and safety. |
| Outcome: | The proposed systems improve system performance, reliability, and safety by integrating human-provided information, feedback, or control into the agent system. |
Educational Dialogue Systems for Visually Impaired Students: Introducing a Task-Oriented User-Agent Corpus (2024.lrec-main)
Copied to clipboard
Elisa Di Nuovo, Manuela Sanguinetti, Pier Felice Balestrucci, Luca Anselma, Cristian Bernareggi, Alessandro Mazzei
| Challenge: | a corpus of real-world dialogues between visually impaired users and an agent is described . the corpus is part of a larger research project aimed at developing tools for easier access to educational content for visually impaired people. |
| Approach: | They describe a corpus consisting of real-world dialogues between users and a task-oriented conversational agent . they report the results of a classification experiment on the annotated corpus and an additional experiment to assess the annotation capabilities of three large language models. |
| Outcome: | The proposed corpus is part of a larger research project aimed at improving visual aids for visually impaired users. |
Human-AI Interaction in the Age of LLMs (2024.naacl-tutorials)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have revolutionized the capabilities of AI systems. |
| Approach: | This tutorial will provide an overview of the interaction between humans and Large Language Models (LLMs) it will start with a review of the types of AI models we interact with and walkthrough of the core concepts in Human-AI Interaction. |
| Outcome: | This tutorial will provide an overview of the interaction between humans and LLMs, exploring the challenges, opportunities, and ethical considerations that arise in this dynamic landscape. |
LLMs syntactically adapt their language use to their conversational partner (2025.acl-short)
Copied to clipboard
| Challenge: | Adapting to the language of a communication partner is associated with increased success in goal-oriented conversations. |
| Approach: | They construct a corpus of conversations between large language models (LLMs) and measure their syntactic adaptation. |
| Outcome: | The proposed model can adapt to the language of the conversational partner in at least a rudimentary way. |
From Word to World: Can Large Language Models be Implicit Text-based World Models? (2026.acl-long)
Copied to clipboard
Yixia Li, Hongru Wang, Jiahao Qiu, Zhenfei Yin, Dongdong Zhang, Cheng Qian, Zeping Li, Xiaoteng Ma, Guanhua Chen, Heng Ji
| Challenge: | Agentic learning increasingly hinges on interaction, yet real-world experience is expensive, limited, and often irreversible at inference time. |
| Approach: | They propose a framework that reframes language modeling as next-state prediction under interaction. |
| Outcome: | The proposed framework evaluates world models in text-based environments . it shows that sufficiently trained models capture coherent environment dynamics . |
I Remember You!: SUI Corpus for Remembering and Utilizing Users’ Information in Chat-oriented Dialogue Systems (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for remembering and utilizing information on users in system utterances do not always fit the context of the dialogue. |
| Approach: | They propose to use user information to fill in utterance templates but the utterrances do not always fit the context. |
| Outcome: | The proposed system can remember and utilize user information on users in dialogues while keeping appropriateness for the context. |
Spoken Conversational Agents with Large Language Models (2025.emnlp-tutorials)
Copied to clipboard
| Challenge: | This tutorial focuses on the evolution of voice-native LLMs . it reviews the adaptation of text LLM to audio, cross-modal alignment, and joint speech–text training . |
| Approach: | This tutorial examines the evolution of voice-native LLMs in conversational agents . it compares cascaded and voice-based LLM systems to end-to-end retrieval-and vision-grounded systems . |
| Outcome: | This tutorial examines the evolution of voice-native LLMs . it compares the performance of voice assistants to current open-domain agents . |
InteractSpeech: A Speech Dialogue Interaction Corpus for Spoken Dialogue Model (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Spoken Dialogue models face challenges in handling nuanced interactional phenomena, such as interruptions and backchannels. |
| Approach: | They propose to use a 150-hour English speech interaction dialogue dataset to empower spoken dialogue models with nuanced real-time interaction capabilities. |
| Outcome: | The proposed dataset trains and evaluates a speech understanding model that classifies key interactional events directly from audio. |
A Semi-autonomous System for Creating a Human-Machine Interaction Corpus in Virtual Reality: Application to the ACORFORMed System for Training Doctors to Break Bad News (L18-1)
Copied to clipboard
Magalie Ochs, Philippe Blache, Grégoire de Montcheuil, Jean-Marie Pergandi, Jorane Saubesty, Daniel Francon, Daniel Mestre
| Challenge: | Existing methods for training doctors to break bad news are expensive and time consuming. |
| Approach: | They propose a method to collect a corpus of human-machine interactions and then construct a semi-autonomous system based on the collected corpus. |
| Outcome: | The proposed system is based on a corpus-based method to analyze human-machine interactions and then develop fully autonomous prototype. |