Every time I fire a conversational designer, the performance of the dialogue system goes down (2022.lrec-1)
Copied to clipboard
Giancarlo Xompero, Michele Mastromattei, Samir Salman, Cristina Giannone, Andrea Favalli, Raniero Romagnoli, Fabio Massimo Zanzotto
| Challenge: | Incorporating handwritten domain scripts into neural-based task-oriented dialogue systems may be an effective way to reduce the need for large sets of annotated dialogues. |
| Approach: | They propose a system where domain scripts are coded in semi-logical rules and evaluated semi-logic rules produced by differently-skilled conversational designers. |
| Outcome: | The proposed system outperforms state-of-the-art systems when trained with smaller sets of annotated dialogues. |
Similar Papers
UniConv: A Unified Conversational Neural Architecture for Multi-domain Task-oriented Dialogues (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to training dialogue agents separately are not optimized for multi-domain task-oriented dialogues. |
| Approach: | They propose a unified neural architecture for end-to-end conversational systems in multi-domain task-oriented dialogues that jointly trains a bi-level state tracker and a joint dialogue act and response generator. |
| Outcome: | The proposed system outperforms existing systems on the MultiWOZ2.1 benchmark in dialogue state tracking, context-to-text, and end-to end settings. |
Addressing Domain Changes in Task-oriented Conversational Agents through Dialogue Adaptation (2023.eacl-srw)
Copied to clipboard
| Challenge: | Recent task-oriented dialogue systems are trained on annotated dialogues, but when domain knowledge changes, the initial model may become obsolete. |
| Approach: | They propose to use an annotated dialogue dataset to train a dialogue model for domain changes . they propose to fine-tune a generative language model on domain changes to reduce performance . |
| Outcome: | The proposed approach reduces performance by 55% by fine-tuning a generative language model on domain changes. |
Semi-Supervised Bootstrapping of Dialogue State Trackers for Task-Oriented Modelling (D19-1)
Copied to clipboard
| Challenge: | Existing systems rely on modular, domain-focused frameworks for analyzing complex problems. |
| Approach: | They propose semi-supervised learning methods that can reduce the amount of required intermediate labelling by leveraging un-annotated data instead of transcribed utterances. |
| Outcome: | The proposed model reduces the amount of turn-level annotations by 30% while maintaining equivalent system performance. |
Building a Role Specified Open-Domain Dialogue System Leveraging Large-Scale Language Models (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent large-scale language models have produced human-like responses in open-domain dialogue systems. |
| Approach: | They propose a framework for imposing roles on open-domain dialogue systems . they use few-shot learning to build a Korean dialogue dataset from scratch . |
| Outcome: | The proposed framework meets role specifications while maintaining conversational abilities. |
Lightweight Transformers for Conversational AI (2022.naacl-industry)
Copied to clipboard
| Challenge: | Commercial dialogue systems typically require a small footprint and fast execution time, but recent trends are in the other direction, resulting in difficulties in model deployment. |
| Approach: | They build Transformer-based Language Models from scratch on large corpora of conversational data and compare their performance against BERT and other strong baselines on dialogue probing tasks. |
| Outcome: | The proposed model outperforms existing models on dialogue probing tasks and can be fine-tuned on a single consumer GPU card. |
Data Collection and End-to-End Learning for Conversational AI (D19-2)
Copied to clipboard
| Challenge: | tutorial aims to familiarise research community with recent advances in statistical dialogue systems . focus of tutorial is on learning end-to-end from data and their relation to more common modular systems. |
| Approach: | This tutorial aims to familiarise the research community with the latest advances in statistical dialogue systems . the focus of the tutorial is on recently introduced end-to-end learning for dialogue systems and their relation to more common modular systems. |
| Outcome: | This tutorial aims to familiarise the research community with the recent advances in statistical dialogue systems for open-domain and task-based dialogue paradigms. |
Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Study (P19-1)
Copied to clipboard
| Challenge: | Neural generative models are becoming more popular when building conversational agents. |
| Approach: | They propose to study the sensitivity of neural dialog models to unnatural perturbations . they experiment with 10 different types of perturbations on 4 multi-turn dialog datasets . |
| Outcome: | The proposed model is sensitive to unnatural changes or perturbations on 4 multi-turn dialog datasets. |
End-to-End Task-Oriented Dialogue Systems Based on Schema (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches for task-oriented dialogue systems rely on a unified schema across domains, but we propose a schema-aware model for task oriented dialogues based on 'slots' |
| Approach: | They propose a schema-aware end-to-end neural network model for handling task-oriented dialogues based on a dynamic set of slots within a unified schema. |
| Outcome: | The proposed model performs better on a well-known dataset than baselines on 'schema-guided dialogue' systems. |
Rationally Reappraising ATIS-based Dialogue Systems (P19-1)
Copied to clipboard
| Challenge: | Recent state-of-the-art neural models have obtained F1-scores near 98% on the task of slot filling. |
| Approach: | They propose to fix annotation errors in ATIS and propose a rule-based grammar for slot filling that achieves a 95.82% F1 score. |
| Outcome: | The proposed grammar achieves a 95.82% F1-score on the ATIS domain. |
Soda-Eval: Open-Domain Dialogue Evaluation in the age of LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Current evaluation practices of open domain dialogue systems are still highly dependent on human evaluation. |
| Approach: | They propose to use an annotated dataset to evaluate chatbots using large language models. |
| Outcome: | The proposed model improves over few-shot inferences on a GPT-3.5 generated dialogue dataset. |