MULTIWOZ

Can Large Language Models Personalize Dialogues to Generational Styles? (2025.findings-emnlp)

Challenge:	Recent studies have found that Task-oriented Dialogue systems can be more suitable for human users.
Approach:	They propose a framework to optimize ToD systems by leveraging Multiple User SimulaTors.
Outcome:	The proposed framework improves performance on multiWOZ with human evaluations and automatic evaluations.

Copied to clipboard

Pier Felice Balestrucci, Ondrej Dusek, Luca Anselma, Alessandro Mazzei

Dialogue State Tracking with Explicit Slot Connection Modeling (2020.acl-main)

Challenge:	a human evaluation reveals that annotators were able to most accurately identify the generation behind P-MultiWoZ dialogues, based only on a single query-reply pair.
Approach:	They create a personalized, generation-specific version of MultiWOZ 2.2 by prompting LLMs to generate personalized dialogue responses.
Outcome:	The proposed model is a personalized version of MultiWOZ 2.2 for Generation X, Y, and Z . it is validated by automatic and human evaluations to determine whether it reflects generational linguistic traits.

Copied to clipboard

Yawen Ouyang, Moxin Chen, Xinyu Dai, Yinggong Zhao, Shujian Huang, Jiajun Chen

Effective and Efficient Conversation Retrieval for Dialogue State Tracking with Implicit Text Summaries (2024.naacl-long)

Challenge:	Existing methods to track dialogue state are lacking in multi-domain scenarios.
Approach:	They propose a model that explicitly considers slot correlations across domains . they propose ellipsis and reference to express values that have been mentioned by slots from other domains.
Outcome:	The proposed model outperforms existing models on multi-domain datasets and achieves state-of-the-art performance.

Copied to clipboard

Seanie Lee, Jianpeng Cheng, Joris Driesen, Alexandru Coca, Anders Johannsen

Dual Slot Selector via Local Reliability Verification for Dialogue State Tracking (2021.acl-long)

Challenge:	Recent studies use in-context learning with large language models (LLM) to find similar dialogue exemplars for prompt learning.
Approach:	They propose to use a conversation retriever to find similar in-context examples for prompt learning.
Outcome:	The proposed approach improves on multiWOZ datasets with GPT-Neo-2.7B and LLaMA-7B/30B .

Copied to clipboard

Jinyu Guo, Kai Shuang, Jijie Li, Zihan Wang

Call, Reward, Repeat: Advancing Dialog State Tracking with GRPO and Function Calling (2026.eacl-srw)

Challenge:	Existing approaches to predict dialogue state from scratch are inefficient and lead to errors . empirical results show that our method achieves 56.93%, 60.73%, and 58.04% joint accuracy on multi-domain conversations .
Approach:	They propose a dual-stage dialogue state tracking method that uses a slot selector and a Slot Value generator to predict the current dialogue state.
Outcome:	The proposed method achieves 56.93%, 60.73%, and 58.04% joint accuracy on multi-domain conversations.

Copied to clipboard

Timur Ionov, Anna Marshalova, Valentin Malykh

Autoregressive Entity Generation for End-to-End Task-Oriented Dialog (2022.coling-1)

Challenge:	Recent advances in Large Language Models (LLMs) have notably enhanced task-oriented dialogue systems, particularly in Dialogue State Tracking (DST).
Approach:	They propose a group-relative policy optimization method that guides LLMs toward improved DST accuracy even under low-resource conditions.
Outcome:	The proposed method improves on established DST benchmarks while using significantly reduced out-of-domain training data.

Copied to clipboard

Guanhuan Huang, Xiaojun Quan, Qifan Wang

Mismatch between Multi-turn Dialogue and its Evaluation Metric in Dialogue State Tracking (2022.acl-short)

Challenge:	Task-oriented dialog systems require external knowledge base to generate a response . current systems require scanning the KB at each turn, which is inefficient when the kb scales up .
Approach:	They propose to generate entity autoregressively before leveraging it to guide response generation.
Outcome:	Experiments on MultiWOZ 2.1 single and CAMREST show that the proposed system generates more high-quality and entity-consistent responses in an end-to-end manner.

Copied to clipboard

Takyoung Kim, Hoonsang Yoon, Yukyung Lee, Pilsung Kang, Misuk Kim

Retrieve & Memorize: Dialog Policy Learning with Multi-Action Memory (2021.findings-acl)

Challenge:	Existing evaluation metrics for dialog state tracking are limited for belief states accumulated as dialog proceeds . relative slot accuracy allows intuitive evaluation by assigning relative scores according to the turn of each dialog .
Approach:	They propose to use relative slot accuracy to complement existing evaluation metrics . joint goal accuracy and slot accuracy are used to evaluate accumulated belief states .
Outcome:	The proposed metrics focus on "penalizing states that fail to predict," not "reward for well-predicted states" the proposed metrics do not depend on the number of predefined slots, and allow intuitive evaluation .

Copied to clipboard

YunHao Li, Yunyi Yang, Xiaojun Quan, Jianxing Yu

Soloist: Building Task Bots at Scale with Transfer Learning and Machine Teaching (2021.tacl-1)

Challenge:	Recent years have seen a rapid growth of interest in building task-oriented dialogue systems.
Approach:	They propose a retrieve-and-memorize framework to deal with unbalanced distribution of system actions in dialogue datasets.
Outcome:	The proposed framework achieves competitive performance among state-of-the-art models on a large-scale task-oriented dialogue dataset.

Copied to clipboard

Baolin Peng, Chunyuan Li, Jinchao Li, Shahin Shayandeh, Lars Liden, Jianfeng Gao

MultiWOZ 2.1: A Consolidated Multi-Domain Dialogue Dataset with State Corrections and State Tracking Baselines (2020.lrec-1)

Challenge:	Existing methods for building task-oriented dialog systems are limited to a few tasks and domains.
Approach:	They propose a method that uses transfer learning and machine teaching to build task bots at scale.
Outcome:	The proposed method outperforms existing methods on well-studied task-oriented dialog benchmarks on well studied tasks.

Copied to clipboard

Mihail Eric, Rahul Goel, Shachi Paul, Abhishek Sethi, Sanchit Agarwal, Shuyang Gao, Adarsh Kumar, Anuj Goyal, Peter Ku, Dilek Hakkani-Tur

Efficient Dialogue State Tracking by Selectively Overwriting Memory (2020.acl-main)

Challenge:	MultiWOZ 2.0 has substantial noise in dialogue state annotations and dialogue utterances . follow-up work has augmented the original dataset with user dialogue acts .
Approach:	They propose to reannotate dialogue state and utterances based on original dataset . they then compare their results to other datasets to improve their models .
Outcome:	The proposed dataset improves on the noise in the dialogue state annotations and dialogue utterances.

Copied to clipboard

Sungdong Kim, Sohee Yang, Gyuwan Kim, Sang-Woo Lee

Learning Low-Resource End-To-End Goal-Oriented Dialog for Fast and Reliable System Deployment (2020.acl-main)

Challenge:	Recent work in dialogue state tracking (DST) is inefficient in that it predicts dialogue state at every turn from scratch.
Approach:	They propose a method that selectively overwrites memory for dialogue state tracking by predicting dialogue state on each memory slot and overwriting it with new values.
Outcome:	The proposed model achieves state-of-the-art joint goal accuracy with 51.72% in MultiWOZ 2.0 and 53.01% in MultiWoz 2.1 in an open vocabulary-based DST setting.

Copied to clipboard

Yinpei Dai, Hangyu Li, Chengguang Tang, Yongbin Li, Jian Sun, Xiaodan Zhu

Paraphrase Augmented Task-Oriented Dialog Generation (2020.acl-main)

Challenge:	Existing end-to-end dialog systems perform less effectively when data is scarce.
Approach:	They propose a Meta-Dialog System which combines meta-learning and human-machine collaboration to improve dialog learning by a new extended-bAbI dataset and a transformed MultiWOZ dataset.
Outcome:	The proposed system outperforms non-meta-learning baselines on a new extended-bAbI dataset and a transformed MultiWOZ dataset for low-resource goal-oriented dialog learning.

Copied to clipboard

Silin Gao, Yichi Zhang, Zhijian Ou, Zhou Yu

Semi-Supervised Dialogue Policy Learning via Stochastic Reward Estimation (2020.acl-main)

Challenge:	Neural generative models can perform dialog generation tasks with a large data set, but lack of high-quality data and expensive data annotation process limit their application in real world settings.
Approach:	They propose to combine paraphrase and response generation models to improve dialog generation performance by annotating dialog states and dialog act labels.
Outcome:	The proposed framework outperforms existing methods significantly in dialog generation tasks, especially under low resource settings.

Copied to clipboard

Xinting Huang, Jianzhong Qi, Yu Sun, Rui Zhang

DS-TOD: Efficient Domain Specialization for Task-Oriented Dialog (2022.findings-acl)

Challenge:	Existing methods for dialogue policy optimization do not provide sufficient supervision signals at the end of dialogues.
Approach:	They propose to learn from state-action pairs of an optimal policy to provide turn-by-turn rewards.
Outcome:	The proposed approach outperforms competitive policy learning baselines on a benchmark multi-domain dataset.

Copied to clipboard

Chia-Chien Hung, Anne Lauscher, Simone Ponzetto, Goran Glavaš

XQA-DST: Multi-Domain and Multi-Lingual Dialogue State Tracking (2023.findings-eacl)

Challenge:	Recent work shows that self-supervised dialog-specific pretraining on large conversational datasets yields substantial gains over traditional language modeling (LM) pretraining.
Approach:	They propose a resource-efficient and modular domain specialization by means of domain adapters in which domain knowledge is encoded.
Outcome:	The proposed framework extracts domain-specific terms and then uses them to build DomainCC and DomainReddit resources based on masked language modeling and response selection objectives.

Copied to clipboard

Han Zhou, Ignacio Iacobacci, Pasquale Minervini

MetaASSIST: Robust Dialogue State Tracking with Meta Learning (2022.emnlp-main)

Challenge:	Existing methods for capturing dialogue data are expensive and limited in their application.
Approach:	They propose a domain-agnostic extractive question answering approach with shared weights across domains to disentangle complex domain information in ToDs.
Outcome:	The proposed model can efficiently leverage domain-agnostic QA datasets while being domain-scalable and open vocabulary in DST.

Copied to clipboard

Fanghua Ye, Xi Wang, Jie Huang, Shenghui Li, Samuel Stern, Emine Yilmaz

Robustness in NLP

Challenge:	Existing dialogue datasets contain lots of noise in their state annotations.
Approach:	They propose a framework to train robust dialogue state tracking models by combining pseudo and vanilla labels by a common weighting parameter.
Outcome:	The proposed framework achieves state-of-the-art accuracy of 80.10% on multiWOZ 2.4.

Value type: the bridge to a better DST model (2023.findings-acl)

Copied to clipboard

Gao Qixiang, Mingyang Sun, Yutao Mou, Chen Zeng, Weiran Xu

Named Entity Recognition

Challenge:	Value type of the slots can provide lots of useful information for DST tasks. however, it has been ignored in most previous works.
Approach:	They propose a new framework for DST task based on slot value type . they propose to extract the type of token from each turn and train a Ner model to extract corresponding type-entity from each conversation according to the token.
Outcome:	The proposed framework is effective on two multi-domain task-oriented conversation datasets.

Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems (P19-1)

Copied to clipboard

Chien-Sheng Wu, Andrea Madotto, Ehsan Hosseini-Asl, Caiming Xiong, Richard Socher, Pascale Fung

Database Search Results Disambiguation for Task-Oriented Dialog Systems (2022.naacl-main)

Challenge:	Existing approaches to dialogue state tracking are dependent on domain ontology and lack of sharing knowledge across domains.
Approach:	They propose a transferable dialogue state generator that generates dialogue states from utterances using copy mechanism.
Outcome:	Empirical results show that TRADE achieves state-of-the-art 48.62% joint goal accuracy for the five domains of MultiWOZ.

Copied to clipboard

Kun Qian, Satwik Kottur, Ahmad Beirami, Shahin Shayandeh, Paul Crook, Alborz Geramifard, Zhou Yu, Chinnadhurai Sankar

Information Retrieval

Challenge:	Task-oriented dialog systems can't handle multiplesearch results when querying a database due to the lack of such scenarios in existing datasets.
Approach:	They propose a task that focuses on disambiguating database search results by synthetically generating turns through a pre-defined grammar and collecting human paraphrases for a subset.
Outcome:	The proposed task improves performance on DSR-disambiguation even in the absence of in-domain data, suggesting it can be learned as a universal dialog skill.

Robust Task-Oriented Dialogue Generation with Contrastive Pre-training and Adversarial Filtering (2022.findings-emnlp)

Copied to clipboard

Shiquan Yang, Xinting Huang, Jey Han Lau, Sarah Erfani

GCDST: A Graph-based and Copy-augmented Multi-domain Dialogue State Tracking (2020.findings-emnlp)

Challenge:	Task-oriented dialogue models can learn non-transferable generalizations by using shortcuts in the data.
Approach:	They propose a contrastive learning framework to encourage models to ignore cues and focus on generalisable patterns.
Outcome:	The proposed framework performs exceptionally well on task-oriented dialogue datasets.

Copied to clipboard

Peng Wu, Bowei Zou, Ridong Jiang, AiTi Aw

Simulated Chats for Building Dialog Systems: Learning to Generate Conversations from Instructions (2021.findings-emnlp)

Challenge:	Existing approaches to training DST on a single domain ignore information across domains.
Approach:	They construct a dialogue state graph to transfer structured features among related domain-slot pairs across domains and encode the graph information of dialogue states by graph convolutional networks.
Outcome:	The proposed model improves the performance of the multi-domain DST baseline with the absolute joint accuracy of 2.0% and 1.0% on the MultiWOZ 2.0 and 2.1 dialogue datasets.

Copied to clipboard

Biswesh Mohapatra, Gaurav Pandey, Danish Contractor, Sachindra Joshi

Alternating Recurrent Dialog Model with Large-scale Pre-trained Language Models (2021.eacl-main)

Challenge:	Popular dialog datasets such as MultiWOZ are created by providing crowd workers with instructions that describe the task to be accomplished.
Approach:	They propose a data creation strategy that uses a pre-trained language model to simulate the interaction between crowd workers by creating a user bot and an agent bot.
Outcome:	The proposed data creation strategy improves on two publicly available datasets using a pre-trained language model and a smaller percentage of actual crowd-generated conversations and their corresponding instructions.

Copied to clipboard

Qingyang Wu, Yichi Zhang, Yu Li, Zhou Yu

Improving End-to-End Task-Oriented Dialog System with A Simple Auxiliary Task (2021.findings-emnlp)

Challenge:	Existing dialog system models require extensive human annotations and are difficult to generalize to different tasks.
Approach:	They propose a framework that uses pre-trained language to model each speaker separately . it can be generalized to more challenging, non-collaborative tasks such as persuasion .
Outcome:	The proposed framework outperforms or is on par with state-of-the-art methods on two popular datasets: CamRest676 and MultiWOZ.

Copied to clipboard

Yohan Lee

Stabilized In-Context Learning with Pre-trained Language Models for Few Shot Dialogue State Tracking (2023.findings-eacl)

Challenge:	Using large pre-trained language models for end-to-end TOD modeling has made significant progress on benchmarks . a paradigm of leveraging large pretrained models has shown promising results .
Approach:	They combine paradigm of leveraging large pre-trained language models with multi-task learning framework . their model achieves new state-of-the-art results with combined scores of 108.3 and 107.5 .
Outcome:	The proposed model achieves state-of-the-art results on multiWOZ 2.0 and MultiWOZ 2.1 . it also improves generalization capability through domain adaptation experiments in the few-shot setting.

Copied to clipboard

Derek Chen, Kun Qian, Zhou Yu

SynthDST: Synthetic Data is All You Need for Few-Shot Dialog State Tracking (2024.eacl-long)

Challenge:	Prompt-based methods with large pre-trained language models have shown impressive unaided performance across many NLP tasks.
Approach:	They propose a meta-learning scheme to stabilize the ability of the model to perform well under various prompts and introduce a saliency model to limit dialogue text length.
Outcome:	The proposed model improves on large pre-trained language models with labeled in-context exemplars and can be used to generate more exemplar queries.

Copied to clipboard

Atharva Kulkarni, Bo-Hsiang Tseng, Joel Moniz, Dhivya Piraviperumal, Hong Yu, Shruti Bhargava

A Unifying View On Task-oriented Dialogue Annotation (2022.lrec-1)

Challenge:	In-context learning with Large Language Models (LLMs) is a promising avenue of research in Dialog State Tracking (DST).
Approach:	They propose a data generation framework tailored for Dialog State Tracking that uses large language models to synthesize natural, coherent, and free-flowing dialogues with DST annotations.
Outcome:	The proposed framework improves joint goal accuracy by 4-5% over the zero-shot baseline on MultiWOZ 2.1 and 2.4.

Copied to clipboard

Vojtěch Hudeček, Leon-paul Schaub, Daniel Stancl, Patrick Paroubek, Ondřej Dušek

HyKnow: End-to-End Task-Oriented Dialog Modeling with Hybrid Knowledge Management (2021.findings-acl)

Challenge:	Recent research attention in task-oriented dialogue systems focuses on end-to-end neural models.
Approach:	They present a dataset that combines annotated corpora from four domains to provide a unified ontology and annotation schema for task-oriented dialogues.
Outcome:	The proposed dataset improves language, information content and performance in dialogues with two recent models.

Copied to clipboard

Silin Gao, Ryuichi Takanobu, Wei Peng, Qun Liu, Minlie Huang

Rethinking Task-Oriented Dialogue Systems: From Complex Modularity to Zero-Shot Autonomous Agent (2024.acl-long)

Challenge:	Task-oriented dialog systems typically manage structured knowledge to guide goal-oriented conversations.
Approach:	They propose a TOD system with hybrid knowledge management, HyKnow, which extends the belief state to manage both structured and unstructured knowledge.
Outcome:	The proposed model outperforms existing TOD systems in the evaluation of a multiWOZ dataset on unstructured knowledge with strong end-to-end performance.

Copied to clipboard

Heng-Da Xu, Xian-Ling Mao, Puhai Yang, Fanshu Sun, Heyan Huang

Exploiting domain-slot related keywords description for Few-Shot Cross-Domain Dialogue State Tracking (2022.emnlp-main)

Challenge:	Task-oriented dialogue systems are designed to be composed of several functional modules, but lacks a general-purpose instruction-following language model.
Approach:	They propose a fully zero-shot autonomous TOD agent that leverages a general-purpose instruction-following language model to decide what to do at each dialogue turn.
Outcome:	The proposed agent can perform tasks in real-life scenarios with a general-purpose instruction-following language model.

Copied to clipboard

Gao Qixiang, Guanting Dong, Yutao Mou, Liwen Wang, Chen Zeng, Daichi Guo, Mingyang Sun, Weiran Xu

BREAK: Breaking the Dialogue State Tracking Barrier with Beam Search and Re-ranking (2023.acl-long)

Challenge:	Existing frameworks for dialogue state tracking with domain-slot-value labels are expensive . current models are limited due to high cost of data annotation and lack of data in some domains .
Approach:	They propose a framework based on domain-slot related description to tackle the challenge of few-shot cross-domain DST.
Outcome:	The proposed framework outperforms existing methods on MultiWOZ and gains strong slot accuracy compared to existing models.

Copied to clipboard

Seungpil Won, Heeyoung Kwak, Joongbo Shin, Janghoon Han, Kyomin Jung

Beyond the Granularity: Multi-Perspective Dialogue Collaborative Selection for Dialogue State Tracking (2022.acl-long)

Challenge:	Existing methods for dialogue state tracking still have a JGA of 60% on MultiWOZ 2.1 . break framework provides a simple yet effective way to generate dialogue state candidates .
Approach:	They propose a framework that generates k-best dialogue state candidates with beam search and re-ranks them to select the correct dialogue state.
Outcome:	The proposed framework pushes the joint goal accuracy to 80-90% on MultiWOZ 2.1-2.4.

Copied to clipboard

Jinyu Guo, Kai Shuang, Jijie Li, Zihan Wang, Yixuan Liu

BORT: Back and Denoising Reconstruction for End-to-End Task-Oriented Dialog (2022.findings-naacl)

Challenge:	Experimental results show that task-oriented dialogue systems have attracted growing attention and achieved substantial progress.
Approach:	They propose a method that dynamically selects relevant dialogue contents for each slot . they retrieve turn-level utterances and evaluate their relevance to the slot from three perspectives .
Outcome:	The proposed method achieves state-of-the-art performance on MultiWOZ 2.1 and MultiWOz 2.2 and superior performance on multiple mainstream benchmark datasets.

Copied to clipboard

Haipeng Sun, Junwei Bao, Youzheng Wu, Xiaodong He

Generation and Extraction Combined Dialogue State Tracking with Hierarchical Ontology Integration (2021.emnlp-main)

Challenge:	Existing task-oriented dialog systems suffer from error propagation from inaccurate dialog states and responses.
Approach:	They propose a back reconstruction approach for end-to-end task-oriented dialog system . they use back reconstruction to reconstruct the original input context from the generated dialog states .
Outcome:	Extensive experiments on MultiWOZ 2.0 and CamRest676 show the effectiveness of the proposed model.

Copied to clipboard

Xinmeng Li, Qian Li, Wansen Wu, Quanjun Yin

Knowledge Representation

Challenge:	Current models are not satisfactory for solving out-of-vocabulary problems . current models assume that the task ontology is well defined in advance .
Approach:	They propose to enhance the interrelation between slots with masked hierarchical attention.
Outcome:	The proposed model yields a significant performance gain over current state-of-the-art model and is more robust to out-ofvocabulary problem compared with other methods.

UNO-DST: Leveraging Unlabelled Data in Zero-Shot Dialogue State Tracking (2024.findings-naacl)

Copied to clipboard

Chuang Li, Yan Zhang, Min-Yen Kan, Haizhou Li

In-Context Learning for Few-Shot Dialogue State Tracking (2022.findings-emnlp)

Challenge:	Existing methods for zero-shot dialogue state tracking (DST) ignore unlabelled data in the target domain.
Approach:	They propose to transform zero-shot dialogue state tracking into few-shot DST by utilising unlabelled data via joint and self-training methods.
Outcome:	The proposed method improves joint goal accuracy by 8% on general language models in zero-shot scenarios, and can be used in many domains.

Copied to clipboard

Yushi Hu, Chia-Hsuan Lee, Tianbao Xie, Tao Yu, Noah A. Smith, Mari Ostendorf

TOD-Flow: Modeling the Structure of Task-Oriented Dialogues (2023.emnlp-main)

Challenge:	Existing methods for zero-shot and few-shot learning dialogue state tracking are hard and expensive.
Approach:	They propose an in-context learning framework for zero-shot and few-shot learning dialogue state tracking (DST) a large pretrained language model takes a test instance and a few exemplars as input and directly decodes the dialogue state .
Outcome:	The proposed framework outperforms state-of-the-art models in few-shot settings . it is flexible and scalable, and requires less data to adapt to new domains and scenarios .

Copied to clipboard

Sungryull Sohn, Yiwei Lyu, Anthony Liu, Lajanugen Logeswaran, Dong-Ki Kim, Dongsub Shim, Honglak Lee

Multi-User MultiWOZ: Task-Oriented Dialogues among Multiple Users (2023.findings-emnlp)

Challenge:	Recent advances in task-oriented dialogue systems have limitations regarding transparency and controllability.
Approach:	They propose to infer the TOD-flow graph from dialog data annotated with dialog acts and integrate it with any dialogue model to improve its prediction performance, transparency, and controllability.
Outcome:	The proposed approach improves dialog act classification and response generation performance in the MultiWOZ and SGD benchmarks.

Copied to clipboard

Yohan Jo, Xinyan Zhao, Arijit Biswas, Nikoletta Basiou, Vincent Auvray, Nikolaos Malandrakis, Angeliki Metallinou, Alexandros Potamianos

ASSIST: Towards Label Noise-Robust Dialogue State Tracking (2022.findings-acl)

Challenge:	a dataset of task-oriented dialogues assume conversations between the agent and one user at a time . but multi-user task-orientated dialogues are richer, containing deliberation and deliberations . a novel task is proposed to rewrite a task-focused query that retains only task-relevant information .
Approach:	They propose to rewrite a task-oriented chat between two users as a concise task-orientated query that retains only task-relevant information and is directly consumable by the dialogue system.
Outcome:	The proposed method surpasses existing models on multi-user dialogues and generalizes to unseen domains.

Copied to clipboard

Fanghua Ye, Yue Feng, Emine Yilmaz

Robustness in NLP

Challenge:	Existing versions of MultiWOZ 2.0 have been published, but there are still lots of noisy labels in the training set.
Approach:	They propose a framework to train dialogue state tracking models from noisy labels instead of improving annotation quality further by using auxiliary models.
Outcome:	The proposed framework improves the goal accuracy of DST models by 28.16% on MultiWOZ 2.0 and 8.41% on MultiWoz 2.4, compared to using only the vanilla noisy labels.

Contextual Dynamic Prompting for Response Generation in Task-oriented Dialog Systems (2023.eacl-main)

Copied to clipboard

Sandesh Swamy, Narges Tabari, Chacha Chen, Rashmi Gangadharaiah

LUNA: Learning Slot-Turn Alignment for Dialogue State Tracking (2022.naacl-main)

Challenge:	Existing studies show that large pre-trained language models can be adapted to task-oriented dialog systems.
Approach:	They propose to use contextual dynamic prompting to generate prompts in dialogs . they propose to distill useful prompting signals from dialog contexts based on contextual dynamic .
Outcome:	The proposed approach improves response generation by 3 points and 17 points when dialog states are incorporated.

Copied to clipboard

Yifan Wang, Jing Zhao, Junwei Bao, Chaoqun Duan, Youzheng Wu, Xiaodong He

Slot Attention with Value Normalization for Multi-Domain Dialogue State Tracking (2020.emnlp-main)

Challenge:	Existing methods exploit the utterances of all dialogue turns to assign value to slots . this can lead to suboptimal results due to information introduced from irrelevant utterrances .
Approach:	They propose a SLot-TUrN Alignment enhanced approach to assign slot value . they explicitly align each slot with its most relevant utterance and then predict the corresponding value based on this aligned utteration.
Outcome:	The proposed approach achieves state-of-the-art on three multi-domain task-oriented dialogue datasets.

Copied to clipboard

Yexiang Wang, Yi Guo, Siqi Zhu

Prompter: Zero-shot Adaptive Prefixes for Dialogue State Tracking Domain Adaptation (2023.acl-long)

Challenge:	Existing dialogue state tracking approaches rely on ontology already defined, where all slots and their possible values are given.
Approach:	They propose a new architecture to exploit domain ontology by using Slot Attention and Value Normalization . they supplement the annotation of supporting span for MultiWOZ 2.1, which is the shortest span in utterances to support the labeled value.
Outcome:	The proposed architecture exploits ontology and can convert supporting spans to values.

Copied to clipboard

Ibrahim Taha Aksu, Min-Yen Kan, Nancy Chen

Multi2WOZ: A Robust Multilingual Dataset and Conversational Pretraining for Task-Oriented Dialog (2022.naacl-main)

Challenge:	Parameter-Efficient Transfer Learning (PETL) has the potential to address this problem, but it has yet to be applied to the zero-shot domain adaptation.
Approach:	They propose to use descriptions of target domain slots to generate dynamic prefixes that are concatenated to the key and values at each layer’s self-attention mechanism.
Outcome:	The proposed method outperforms previous methods on the MultiWOZ and SGD benchmarks.

Copied to clipboard

Chia-Chien Hung, Anne Lauscher, Ivan Vulić, Simone Ponzetto, Goran Glavaš

MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems (2020.emnlp-main)

Challenge:	Task-oriented dialog (TOD) is arguably one of the most popular natural language processing (NLP) application areas.
Approach:	They propose a multilingual multi-domain TOD dataset that spans four languages . they use a framework for multilingual conversational specialization of pretrained language models .
Outcome:	The proposed datasets show that they perform better than existing datasets in English . the proposed framework allows for sample-efficient few-shot transfer for TOD tasks .

Copied to clipboard

Zhaojiang Lin, Andrea Madotto, Genta Indra Winata, Pascale Fung

Multi-Domain Dialogue State Tracking with Disentangled Domain-Slot Attention (2023.findings-acl)

Challenge:	Existing approaches to learn dialogue state tracking and response generation are time-intensive and not transferable between domains.
Approach:	They propose a transfer learning framework that allows efficient dialogue state tracking with a minimal generation length.
Outcome:	The proposed framework improves the inference efficiency and improves state-of-the-art results on multi-domain multi-tasking systems.

Copied to clipboard

Longfei Yang, Jiyi Li, Sheng Li, Takahiro Shinozaki

Zero-Shot Cross-Domain Dialogue State Tracking via Dual Low-Rank Adaptation (2024.acl-long)

Challenge:	Multi-domain dialogue state tracking is a challenge for task-oriented dialogue systems . domains and slots are aggregated into a single query to generate domain-slot specific representations .
Approach:	They propose to disentangle domain-slot attention for multi-domain dialogue state tracking by separating query about domains and slots from the attention component.
Outcome:	The proposed approach outperforms the standard multi-head attention with aggregated domain-slot query.

Copied to clipboard

Xiang Luo, Zhiwen Tang, Jin Wang, Xuejie Zhang

A Few-Shot Semantic Parser for Wizard-of-Oz Dialogues with the Precise ThingTalk Representation (2022.findings-acl)

Challenge:	Existing approaches to zero-shot dialogue state tracking (DST) involve embedding prompts into language models, but these methods have inherent limitations.
Approach:	They propose a plug-and-play architecture designed for zero-shot dialogue state tracking (DST) dual low-rank adaptation targets dialogue context processing and prompt optimization without incurring additional inference latency.
Outcome:	The proposed architecture outperforms baseline methods on multi-domain datasets and the MultiWOZ dataset.

Copied to clipboard

Giovanni Campagna, Sina Semnani, Ryan Kearns, Lucas Jun Koba Sato, Silei Xu, Monica Lam

Semantic Text Processing

Challenge:	Existing approaches to build effective semantic parsers for Wizard-of-Oz are insufficient.
Approach:	They propose a new dialogue representation and a sample-efficient methodology that can predict precise dialogue states in WOZ conversations.
Outcome:	The proposed model can predict precise dialogue states in WOZ conversations.

Show, Don’t Tell: Demonstrations Outperform Descriptions for Schema-Guided Task-Oriented Dialogue (2022.naacl-main)

Copied to clipboard

Raghav Gupta, Harrison Lee, Jeffrey Zhao, Yuan Cao, Abhinav Rastogi, Yonghui Wu

Diverse Retrieval-Augmented In-Context Learning for Dialogue State Tracking (2023.findings-acl)

Challenge:	Recent work has leveraged natural language descriptions of schema elements to enable universal dialogue systems; however, descriptions only indirectly convey schema semantics.
Approach:	They propose to use schema-guided modeling to prompt seq2seq models with a labeled example dialogue to show schema semantics rather than tell them.
Outcome:	The proposed model outperforms models using short examples as schema representations on two popular dialogue state tracking benchmarks.

Copied to clipboard

Brendan King, Jeffrey Flanigan

Generalizable and Explainable Dialogue Generation via Explicit Action Learning (2020.findings-emnlp)

Challenge:	Recent work has demonstrated that in-context learning for dialogue state tracking outperforms training methods in the few-shot setting.
Approach:	They propose a method for in-context learning for dialogue state tracking that takes into account probabilities of competing surface forms and produces a more accurate dialogue state prediction.
Outcome:	The proposed method outperforms trained methods in the few-shot setting and requires little data and zero parameter updates.

Copied to clipboard

Xinting Huang, Jianzhong Qi, Yu Sun, Rui Zhang

Dialogue State Tracking with a Language Model using Schema-Driven Prompting (2021.emnlp-main)

Challenge:	Conditioned response generation for task-oriented dialogues implicitly optimizes task completion and language quality.
Approach:	They propose to learn natural language actions that represent utterances as a span of words.
Outcome:	The proposed approach outperforms latent action baselines on a multi-domain dataset.

Copied to clipboard

Chia-Hsuan Lee, Hao Cheng, Mari Ostendorf

Plan, Generate and Complicate: Improving Low-resource Dialogue State Tracking via Easy-to-Difficult Zero-shot Data Augmentation (2024.findings-acl)

Challenge:	Task-oriented conversational systems often use dialogue state tracking to represent the user’s intentions, which involves filling in values of pre-defined slots.
Approach:	They propose a schema-driven prompting approach that provides task-aware history encoding that is used for both categorical and non-categorical slots.
Outcome:	The proposed system achieves state-of-the-art performance on MultiWOZ 2.2 and competitive performance on two other benchmarks: MultiWOz 2.1 and M2M.

Copied to clipboard

Ming Gu, Yan Yang

Comparing Data Augmentation Methods for End-to-End Task-Oriented Dialog Systems (2024.findings-acl)

Challenge:	Existing data augmentation methods rely on pre-defined user goals and neglect the importance of data complexity in this task.
Approach:	They propose an Easy-to-Difficult Zero-shot Data Augmentation framework for low-resource dialogue state tracking that utilizes large language models to automatically catch the relationships of different domains and then generate dialogue data.
Outcome:	The proposed framework outperforms existing methods on MultiWOZ and other similar datasets.

Copied to clipboard

Christos Vlachos, Themos Stafylakis, Ion Androutsopoulos

EmoWOZ: A Large-Scale Corpus and Labelling Scheme for Emotion Recognition in Task-Oriented Dialogue Systems (2022.lrec-1)

Challenge:	Creating effective task-oriented dialog systems is challenging due to the scarcity of training data.
Approach:	They empirically evaluate eight DA methods that have shown promising results in task-oriented dialog systems and other NLP systems.
Outcome:	The proposed methods have been successful in other NLP systems but not in the ToDSs.

Copied to clipboard

Shutong Feng, Nurul Lubis, Christian Geishauser, Hsien-chin Lin, Michael Heck, Carel van Niekerk, Milica Gasic

Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue StateTracking (2021.naacl-main)

Challenge:	Existing emotion-annotated task-oriented corpora are limited in size, label richness, and public availability, creating a bottleneck for downstream tasks.
Approach:	They propose a large-scale manually emotion-annotated corpus of task-oriented dialogues based on a multi-domain task-orientated dataset.
Outcome:	The proposed method is based on a task-oriented dialogue dataset with 11K dialogues and 83K emotion annotations of user utterances.

Copied to clipboard

Zhaojiang Lin, Bing Liu, Seungwhan Moon, Paul Crook, Zhenpeng Zhou, Zhiguang Wang, Zhou Yu, Andrea Madotto, Eunjoon Cho, Rajen Subba

Multi-Domain Goal-Oriented Dialogues (MultiDoGO): Strategies toward Curating and Annotating Large Scale Dialogue Data (D19-1)

Challenge:	Existing models for zero-shot cross-domain dialogue state tracking require in-domain data to model a new domain.
Approach:	They propose a slot descriptions enhanced generative approach for zero-shot cross-domain DST by encoding a dialogue context and a slots with a pre-trained encoder and generating slot value in auto-regressive manner.
Outcome:	The proposed model significantly improves state-of-the-art results in zero-shot cross-domain setting.

Copied to clipboard

Denis Peskov, Nancy Clarke, Jason Krone, Brigi Fodor, Yi Zhang, Adel Youssef, Mona Diab

Task-Optimized Adapters for an End-to-End Task-Oriented Dialogue System (2023.findings-acl)

Challenge:	a large number of goal-oriented dialogue datasets are limited in their size, linguistic diversity, domain coverage, or annotation granularity.
Approach:	They propose a multi-domain goal-oriented dialogue dataset that uses a crowd-sourced worker and a trained annotator to curate and annotate large scale data.
Outcome:	The proposed dataset is 8 times the size of the largest comparable dialogue dataset available to the public.

Copied to clipboard

Namo Bang, Jeehyun Lee, Myoung-Wan Koo

Scalable-DSC: A Structural Template Prompt Approach to Scalable Dialogue State Correction (2023.emnlp-main)

Challenge:	Recent work on end-to-end dialogue models with pre-trained dialogue corpora shows promising performance in the conversational system.
Approach:	They propose an end-to-end TOD system with task-optimized adapters which learn independently per task adding only small number of parameters after fixed layers of pre-trained network.
Outcome:	The proposed system achieves state-of-the-art performance on the MultiWOZ benchmark compared to existing models.

Copied to clipboard

Haoxiang Su, Hongyan Xie, Hao Huang, Shuangyong Song, Ruiyu Fang, Xiaomeng Huang, Sijie Feng

SUMBT: Slot-Utterance Matching for Universal and Scalable Belief Tracking (P19-1)

Challenge:	Existing approaches to correct wrong slot values in dialogue state tracking are intertwined with specific DST models, limiting their applicability to other DSTs.
Approach:	They propose a Scalable Dialogue State Correction model that corrects wrong slot values in predicted dialogue states by using a structural template prompt.
Outcome:	The proposed model achieves state-of-the-art results on MultiWOZ 2.0-2.4.

Copied to clipboard

Hwaran Lee, Jinsik Lee, Tae-Yoon Kim

MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling (D18-1)

Challenge:	Existing methods to model domain- and slot-dependent belief trackers have difficulty adding new slot-values, resulting in lack of flexibility of domain ontology configurations.
Approach:	They propose a model that captures relationships between domain-slot-types and slot-values appearing in utterances through attention mechanisms based on contextual semantic vectors.
Outcome:	The proposed model improves performance on two dialog corpora and achieves state-of-the-art accuracy.

Copied to clipboard

Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, Milica Gašić

A Contextual Hierarchical Attention Network with Adaptive Objective for Dialogue State Tracking (2020.acl-main)

Challenge:	a dataset of 10k human-human written conversations is one order of magnitude larger than previous annotated task-oriented corpora.
Approach:	They propose to collect 10k human-human written conversations from a crowd-sourced dataset using crowd-sourcing.
Outcome:	The proposed dataset is one order of magnitude larger than previous annotated task-oriented corpora and shows the usability of the data and sets a baseline for future studies.

Copied to clipboard

Yong Shan, Zekang Li, Jinchao Zhang, Fandong Meng, Yang Feng, Cheng Niu, Jie Zhou

ResFormer: All-Time Reservoir Memory for Long Sequence Classification (2025.emnlp-main)

Challenge:	Existing methods for dialogue state tracking ignore the slot imbalance problem and treat all slots indiscriminately, which limits the learning of hard slots.
Approach:	They propose to employ a contextual hierarchical attention network to enhance the DST by learning contextual representations.
Outcome:	The proposed approach achieves 52.68% and 58.55% joint accuracy on multiWOZ 2.0 and MultiWOZ 2.1 datasets and significantly improves performance (+1.24% and +5.98%)

Copied to clipboard

Hongbo Liu, Jia Xu

Information Retrieval

Challenge:	Existing models with quadratic time and memory complexity restrict input length . however, analyzing extensive sequential contexts is challenging .
Approach:	They propose a neural network architecture that captures contextual dependencies in linear time and a nonlinear readout to model short-term dependencies within sentences.
Outcome:	The proposed model outperforms baseline models on EmoryNLP datasets and on IEMOCAP and MultiWOZ datasets.

Can a Single Model Master Both Multi-turn Conversations and Tool Use? CoALM: A Unified Conversational Agentic Language Model (2025.acl-long)

Copied to clipboard

Emre Can Acikgoz, Jeremiah Greer, Akul Datta, Ze Yang, William Zeng, Oussama Elachqar, Emmanouil Koukoumidis, Dilek Hakkani-Tür, Gokhan Tur

Knowledge-Aware Graph-Enhanced GPT-2 for Dialogue State Tracking (2021.emnlp-main)

Challenge:	Large Language Models (LLMs) with API-calling capabilities enabled building effective Language Agents (LA) current approaches excel in one domain but underperform in the other.
Approach:	They propose a unified approach that integrates both conversational and agentic capabilities.
Outcome:	The proposed model outperforms top domain-specific models across three benchmarks.

Copied to clipboard

Weizhe Lin, Bo-Hsiang Tseng, Bill Byrne

Comet: Dialog Context Fusion Mechanism for End-to-End Task-Oriented Dialog with Multi-task Learning (2025.coling-main)

Challenge:	Existing models for dialogue state tracking are based on Graph Attention Networks . if the relationship between slots and values is modelled explicitly, this can be improved .
Approach:	They propose a model architecture that augments GPT-2 with Graph Attention Networks to allow sequential prediction of slot values.
Outcome:	The proposed architecture improves performance against a strong GPT-2 baseline and with sparsely supervised training.

Copied to clipboard

Haipeng Sun, Junwei Bao, Youzheng Wu, Xiaodong He

Enhancing AI Assisted Writing with One-Shot Implicit Negative Feedback (2024.emnlp-main)

Challenge:	Existing end-to-end task-oriented dialog systems often encounter challenges arising from implicit information, coreference, and the presence of noisy and irrelevant data within the dialog context.
Approach:	They propose a dialog context fusion mechanism for end-to-end task-oriented dialog augmented with three additional tasks: dialog summarization, domain prediction, and slot detection.
Outcome:	The proposed method achieves state-of-the-art on the MultiWOZ and CrossWOZ datasets.

Copied to clipboard

Benjamin Towle, Ke Zhou

Challenge:	Various systems have been proposed to draft and automate replies for users . yet, the heterogeneity of the inputs and architectures often renders it difficult to combine insights from user behaviour in one system to improve performance in another.
Approach:	They propose an approach that uses classifier guidance to controllably integrate implicit user feedback into the text generation process.
Outcome:	The proposed approach improves Rouge-L, generating the correct intent and generating an 86% win-rate on the multiWOZ and Schema-Guided Dialog datasets.

Mars: Modeling Context & State Representations with Contrastive Learning for End-to-End Task-Oriented Dialog (2023.findings-acl)

Copied to clipboard

Haipeng Sun, Junwei Bao, Youzheng Wu, Xiaodong He

Semantic Text Processing

Challenge:	Empirical results show dialog context representations are more conducive to multi-turn task-oriented dialog.
Approach:	They propose an end-to-end task-oriented dialog system with two contrastive learning strategies to model relationship between dialog context and belief/action state representations.
Outcome:	Empirical results show that dialog context representations are more conducive to multi-turn task-oriented dialog.

A Probabilistic End-To-End Task-Oriented Dialog Model with Latent Belief States towards Semi-Supervised Learning (2020.emnlp-main)

Copied to clipboard

Yichi Zhang, Zhijian Ou, Min Hu, Junlan Feng

Semantic Parsing by Large Language Models for Intricate Updating Strategies of Zero-Shot Dialogue State Tracking (2023.findings-emnlp)

Challenge:	Structured belief states are crucial for goal tracking and database query in task-oriented dialog systems.
Approach:	They propose a probabilistic dialog model where belief states are represented as discrete latent variables and jointly modeled with system responses given user inputs.
Outcome:	The proposed model outperforms supervised-only and semi-supervised baselines on three benchmark datasets.

Copied to clipboard

Yuxiang Wu, Guanting Dong, Weiran Xu

Semantic Text Processing

Challenge:	Existing methods for zero-shot Dialogue State Tracking have focused on domaintransfers and have not yielded satisfactory results.
Approach:	They propose a new In-Context Learning method to introduce additional updating strategies in zero-shot DST by leveraging powerful Large Language Models and translating the original dialogue to JSON through semantic parsing as an intermediate state.
Outcome:	The proposed method outperforms existing zero-shot DST methods on MultiWOZ, showing significant improvements in JGA and slot accuracy compared to existing methods.

AnyTOD: A Programmable Task-Oriented Dialog System (2023.emnlp-main)

Copied to clipboard

Jeffrey Zhao, Yuan Cao, Raghav Gupta, Harrison Lee, Abhinav Rastogi, Mingqiu Wang, Hagen Soltau, Izhak Shafran, Yonghui Wu

Navigating Rifts in Human-LLM Grounding: Study and Benchmark (2025.acl-long)

Challenge:	a neuro-symbolic approach allows zero-shot adaptation to unseen tasks and domains . a neural LM keeps track of events that occur during a conversation and a symbolic program implements dialog policy is executed to recommend actions.
Approach:	They propose an end-to-end, zero-shot task-oriented dialog system . it is designed to adapt to unseen tasks or domains without prior training .
Outcome:	The proposed system can be programmed to adapt to unseen tasks without training . it reduces data collection and training requirements for enabling new TOD 1 16189 tasks .

Copied to clipboard

Omar Shaikh, Hussein Mozannar, Gagan Bansal, Adam Fourney, Eric Horvitz

Challenge:	Language models excel at following instructions but struggle with collaborative aspects of conversation that humans naturally employ.
Approach:	They analyze logs from WildChat, MultiWOZ, and Bing Chat to examine grounding challenges . they propose a benchmark to determine when LLMs fail to initiate grounding .
Outcome:	The proposed model predicts interactions that fail to ground with users . the proposed model is based on human-human interactions with humans .

Sparse Rewards Can Self-Train Dialogue Agents (2025.findings-acl)

Copied to clipboard

Barrett Martin Lattimer, Varun Prashant Gangal, Ryan McDonald, Yi Yang

Improving Dialogue State Tracking through Combinatorial Search for In-Context Examples (2025.acl-long)

Challenge:	Recent advances in large language models have been driven by supervised fine-tuning and high-quality human feedback. however, acquiring meaningful human feedback has become increasingly challenging and costly.
Approach:	They propose a method that empowers LLM agents to enhance their performance without external feedback.
Outcome:	The proposed method improves tool-based interactions while preserving general model capabilities across diverse benchmarks.

Copied to clipboard

Haesung Pyun, Yoonah Park, Yohan Jo

Information Retrieval

Challenge:	Existing methods for training dialogue state tracking data are suboptimal . existing methods rely on suboptimized data, resulting in poor performance .
Approach:	They propose a method that scores effective in-context examples based on their combinatorial impact on DST performance.
Outcome:	The proposed method achieves a 20% gain in data efficiency and generalizing well to the SGD dataset.

Know Your Mistakes: Towards Preventing Overreliance on Task-Oriented Conversational AI Through Accountability Modeling (2025.acl-long)

Copied to clipboard

Suvodip Dey, Yi-Jyun Sun, Gokhan Tur, Dilek Hakkani-Tür

Towards Cost-effective Multi-style Conversations: A Pilot Study in Task-oriented Dialogue Generation (2024.lrec-main)

Challenge:	Recent LLMs are known to hallucinate, producing responses that seem plausible but are factually incorrect.
Approach:	They propose an accountability model for LLM-based task-oriented dialogue agents to address user overreliance via friction turns in cases of model uncertainty and errors associated with dialogue state tracking (DST).
Outcome:	The proposed model improves joint goal accuracy (JGA) of DST output by 3% on two established benchmarks.

Copied to clipboard

Tiziano Labruna, Bernardo Magnini

Reading Between the Lines: The One-Sided Conversation Problem (2026.findings-acl)

Challenge:	Current task-oriented dialogue systems are trained on a single conversational style and do not account for the diversity of styles encountered when interacting with different users.
Approach:	They propose a method for generating multi-style conversations using a multi-language dataset that is available in a conversational domain.
Outcome:	The proposed model can be used in the development of conversational agents . it assumes the availability of a conversational domain and leverages the generative capabilities of large language models.

Copied to clipboard

Victoria Ebert, Rishabh Singh, Tuochao Chen, Noah A. Smith, Shyamnath Gollakota

HiCoLoRA: Addressing Context-Prompt Misalignment via Hierarchical Collaborative LoRA for Zero-Shot DST (2026.findings-acl)

Challenge:	In many real-world scenarios, only one side of a conversation is available for processing.
Approach:	They propose a one-sided conversation problem to reconstruct the missing speaker's turns and generate faithful summaries from one-side transcripts.
Outcome:	The proposed model improves reconstructions with prompting, but smaller models require fine tuning.

Copied to clipboard

Shuyu Zhang, Yifan Wei, Xinru Wang, Yanmin Zhu, Yangfan He, Yixuan Weng, Yujie Liu, Bin Li