ADVISER: A Toolkit for Developing Multi-modal, Multi-domain and Socially-engaged Conversational Agents (2020.acl-demos)
Copied to clipboard
Chia-Yu Li, Daniel Ortega, Dirk Väth, Florian Lux, Lindsey Vanderlyn, Maximilian Schmidt, Michael Neumann, Moritz Völkel, Pavel Denisov, Sabrina Jenne, Zorica Kacarevic, Ngoc Thang Vu
| Challenge: | Existing toolkits for developing dialog systems are limited to core components and do not support multi-modal processing and social signals. |
| Approach: | They propose to use ADVISER to develop multi-modal dialog agents using multi-text and social signals. |
| Outcome: | The proposed toolkit is flexible, easy to use, and easy to extend for linguists and cognitive scientists, thereby providing a flexible platform for collaborative research. |
Similar Papers
ADVISER: A Dialog System Framework for Education & Research (P19-3)
Copied to clipboard
Daniel Ortega, Dirk Väth, Gianna Weber, Lindsey Vanderlyn, Maximilian Schmidt, Moritz Völkel, Zorica Karacevic, Ngoc Thang Vu
| Challenge: | In this paper, we focus on task-oriented dialog systems, although our framework allows easy integration of non-task dialog systems and their combination. |
| Approach: | They propose an open source dialog system framework for education and research that supports multi-domain task-oriented conversations in two languages. |
| Outcome: | The proposed framework supports multi-domain task-oriented conversations in two languages and is open source for education and research. |
SDialog: A Python Toolkit for End-to-End Agent Building, User Simulation, Dialog Generation, and Evaluation (2026.eacl-demo)
Copied to clipboard
Sergio Burdisso, Séverin Baroudi, Yanis Labrak, David Grünert, Pawel Cyrta, Yiyang Chen, Srikanth Madikeri, Esaú Villatoro-tello, Ricard Marxer, Petr Motlicek
| Challenge: | SDialog is an open-source Python toolkit for end-to-end development, simulation, evaluation and analysis of LLM-based conversational agents. |
| Approach: | They present an open-source Python toolkit for end-to-end development, simulation, evaluation and analysis of LLM-based conversational agents. |
| Outcome: | SDialog enables more controlled, transparent, and systematic research on conversational systems. |
PyOpenDial: A Python-based Domain-Independent Toolkit for Developing Spoken Dialogue Systems with Probabilistic Rules (D19-3)
Copied to clipboard
| Challenge: | a recent development of spoken dialogue systems has enabled deep learning to achieve state-of-the-art performance. |
| Approach: | They propose a Python-based domain-independent, open-source toolkit for spoken dialogue systems. |
| Outcome: | The proposed toolkit extends OpenDial's Java-based architecture and provides new functions for neural dialogue state tracking and action planning. |
ProsocialDialog: A Prosocial Backbone for Conversational Agents (2022.emnlp-main)
Copied to clipboard
Hyunwoo Kim, Youngjae Yu, Liwei Jiang, Ximing Lu, Daniel Khashabi, Gunhee Kim, Yejin Choi, Maarten Sap
| Challenge: | Existing dialogue systems fail to respond properly to potentially unsafe user utterances . existing systems either ignore or passively agree with unsafe content . |
| Approach: | They introduce a dataset to teach conversational agents to respond to problematic content following social norms. |
| Outcome: | The proposed dataset shows that ProsocialDialog generates more socially acceptable dialogues than existing models. |
Multi-Modal Open-Domain Dialogue (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work in open-domain conversational agents has demonstrated that significant improvements in humanness and user preference can be achieved via massive scaling in both pre-training data and model size. |
| Approach: | They combine open-domain dialogue agents with vision models to investigate human preferences and humanness. |
| Outcome: | The proposed model outperforms existing models in multi-modal dialogue while performing as well as its predecessor (text-only) BlenderBot. |
The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agents (2020.acl-main)
Copied to clipboard
| Challenge: | a set of 12 tasks that measure if a conversational agent can communicate engagingly with personality and empathy, ask questions, answer questions by utilizing knowledge resources, and perceive and converse about images. |
| Approach: | They propose a set of 12 tasks that measure if a conversational agent can communicate engagingly with personality and empathy . they use large dialogue datasets to multi-task and obtain state-of-the-art results . |
| Outcome: | The proposed model improves over a BERT pre-trained model on large dialogue datasets and provides state-of-the-art results on many of the tasks. |
DeepPavlov: Open-Source Library for Dialogue Systems (P18-4)
Copied to clipboard
Mikhail Burtsev, Alexander Seliverstov, Rafael Airapetyan, Mikhail Arkhipov, Dilyara Baymurzina, Nickolay Bushkov, Olga Gureenkova, Taras Khakhulin, Yuri Kuratov, Denis Kuznetsov, Alexey Litinsky, Varvara Logacheva, Alexey Lymar, Valentin Malykh, Maxim Petrov, Vadim Polulyakh, Leonid Pugachev, Alexey Sorokin, Maria Vikhreva, Marat Zaynutdinov
| Challenge: | open-source library DeepPavlov is designed for rapid development of dialogue systems. |
| Approach: | open-source library DeepPavlov is tailored for development of conversational agents . the library prioritizes efficiency, modularity and extensibility with the goal to make it easier to develop dialogue systems from scratch . |
| Outcome: | the open-source library DeepPavlov is designed for rapid development of dialogue systems . it supports modular as well as end-to-end approaches to implementation of conversational agents . |
AgentMaster: A Multi-Agent Conversational Framework Using A2A and MCP Protocols for Multimodal Information Retrieval and Analysis (2025.emnlp-demos)
Copied to clipboard
| Challenge: | Recent advances in AI focus on multi-agent systems (MAS) that can be integrated with Large Language Models (LLMs) but current systems still face challenges of inter-agency communication, coordination, and interaction with heterogeneous tools and resources. |
| Approach: | They propose a modular multi-protocol MAS framework with self-implemented A2A and MCP . the framework supports natural language interaction without prior technical expertise . |
| Outcome: | The proposed framework supports natural language interaction without prior technical expertise and responds to multimodal queries for tasks including information retrieval, question answering, and image analysis. |
Extending Neural Generative Conversational Model using External Knowledge Sources (D18-1)
Copied to clipboard
| Challenge: | Existing generative dialogue models lack coherence and are content poor . however, current models lack the capacity to handle large unstructured knowledge sources. |
| Approach: | They propose an architecture to incorporate unstructured knowledge sources to enhance the next utterance prediction in chit-chat type of generative dialogue models. |
| Outcome: | The proposed architecture improves the next utterance prediction in chit-chat type of generative dialogue models by incorporating external knowledge from Wikipedia summaries and the NELL knowledge base. |
Can You Put it All Together: Evaluating Conversational Agents’ Ability to Blend Skills (2020.acl-main)
Copied to clipboard
| Challenge: | Existing work has focused on learning specific qualities of conversational agents, but it remains unclear how to combine them. |
| Approach: | They propose to combine models trained towards isolated capabilities with multi-task training to improve conversation performance. |
| Outcome: | The proposed dataset compares models trained towards isolated capabilities with models trained on a single skill. |