DST

Effective and Efficient Conversation Retrieval for Dialogue State Tracking with Implicit Text Summaries (2024.naacl-long)

Challenge:	Existing methods to track dialogue state are lacking in multi-domain scenarios.
Approach:	They propose a model that explicitly considers slot correlations across domains . they propose ellipsis and reference to express values that have been mentioned by slots from other domains.
Outcome:	The proposed model outperforms existing models on multi-domain datasets and achieves state-of-the-art performance.

Copied to clipboard

Seanie Lee, Jianpeng Cheng, Joris Driesen, Alexandru Coca, Anders Johannsen

Dynamic Schema Graph Fusion Network for Multi-Domain Dialogue State Tracking (2022.acl-long)

Challenge:	Recent studies use in-context learning with large language models (LLM) to find similar dialogue exemplars for prompt learning.
Approach:	They propose to use a conversation retriever to find similar in-context examples for prompt learning.
Outcome:	The proposed approach improves on multiWOZ datasets with GPT-Neo-2.7B and LLaMA-7B/30B .

Copied to clipboard

Yue Feng, Aldo Lipani, Fanghua Ye, Qiang Zhang, Emine Yilmaz

Dual Slot Selector via Local Reliability Verification for Dialogue State Tracking (2021.acl-long)

Challenge:	Existing approaches to model the relations between domains and slots fail to address these issues and can be generalized to unseen domains.
Approach:	They propose a Dynamic Schema Graph Fusion Network which generates a dynamic schema graph to explicitly fuse prior slot-domain membership relations and dialogue-aware dynamic slot relations.
Outcome:	The proposed model outperforms existing methods on benchmark datasets showing that it can extract users' goals or intentions as dialogue states and keep them updated over the whole dialogue.

Copied to clipboard

Jinyu Guo, Kai Shuang, Jijie Li, Zihan Wang

Multi-Domain Dialogue State Tracking By Neural-Retrieval Augmentation (2022.findings-aacl)

Challenge:	Existing approaches to predict dialogue state from scratch are inefficient and lead to errors . empirical results show that our method achieves 56.93%, 60.73%, and 58.04% joint accuracy on multi-domain conversations .
Approach:	They propose a dual-stage dialogue state tracking method that uses a slot selector and a Slot Value generator to predict the current dialogue state.
Outcome:	The proposed method achieves 56.93%, 60.73%, and 58.04% joint accuracy on multi-domain conversations.

Copied to clipboard

Lohith Ravuru, Seonghan Ryu, Hyungtak Choi, Haehun Yang, Hyeonmok Ko

Find or Classify? Dual Strategy for Slot-Value Predictions on Multi-Domain Dialog State Tracking (2020.starsem-1)

Challenge:	Existing approaches for DST are conditioned on previous dialogue states, but the dependency on previous dialogs makes it difficult to prevent error propagation to subsequent turns.
Approach:	They propose to create a Neural Index based on dialogue context by analyzing user dialogue and previous turn state and generating a retrieval-guided generation approach.
Outcome:	The proposed framework retrieves dialogue context from the index built using unstructured dialogue state and structured user/system utterances.

Copied to clipboard

Jianguo Zhang, Kazuma Hashimoto, Chien-Sheng Wu, Yao Wang, Philip Yu, Richard Socher, Caiming Xiong

Fully Statistical Neural Belief Tracking (P18-2)

Challenge:	Existing methods for dialog state tracking are ontology-based and ontologie-free . however, it is not clear enough which slots are better handled by either of the two methods .
Approach:	They propose a dual-strategy model that integrates both ontology-based and ontological-free methods.
Outcome:	The proposed model outperforms the existing model on noisy and cleaner datasets.

Copied to clipboard

Nikola Mrkšić, Ivan Vulić

Call, Reward, Repeat: Advancing Dialog State Tracking with GRPO and Function Calling (2026.eacl-srw)

Challenge:	Existing framework for a dialogue state tracking model requires an expensive manual retuning step .
Approach:	They propose to improve existing NBT model by removing a manual retuning step . they propose two different statistical update mechanisms to improve model performance .
Outcome:	The proposed model achieves competitive performance and provides a robust framework for building resource-light DST models.

Copied to clipboard

Timur Ionov, Anna Marshalova, Valentin Malykh

Schema Encoding for Transferable Dialogue State Tracking (2022.coling-1)

Challenge:	Recent advances in Large Language Models (LLMs) have notably enhanced task-oriented dialogue systems, particularly in Dialogue State Tracking (DST).
Approach:	They propose a group-relative policy optimization method that guides LLMs toward improved DST accuracy even under low-resource conditions.
Outcome:	The proposed method improves on established DST benchmarks while using significantly reduced out-of-domain training data.

Copied to clipboard

Hyunmin Jeon, Gary Geunbae Lee

Mismatch between Multi-turn Dialogue and its Evaluation Metric in Dialogue State Tracking (2022.acl-short)

Challenge:	Recent work has focused on deep neural models for task-oriented dialogue systems . however, the neural models require a large dataset for training and a new dataset to be trained on another domain.
Approach:	They propose a schema encoder for transferable dialogue state tracking to new domains . they aim to transfer the model to new datasets by encoding new schemas based on the dataset .
Outcome:	The proposed method improves the accuracy of the proposed model on multi-domain settings.

Copied to clipboard

Takyoung Kim, Hoonsang Yoon, Yukyung Lee, Pilsung Kang, Misuk Kim

MoNET: Tackle State Momentum via Noise-Enhanced Training for Dialogue State Tracking (2023.findings-acl)

Challenge:	Existing evaluation metrics for dialog state tracking are limited for belief states accumulated as dialog proceeds . relative slot accuracy allows intuitive evaluation by assigning relative scores according to the turn of each dialog .
Approach:	They propose to use relative slot accuracy to complement existing evaluation metrics . joint goal accuracy and slot accuracy are used to evaluate accumulated belief states .
Outcome:	The proposed metrics focus on "penalizing states that fail to predict," not "reward for well-predicted states" the proposed metrics do not depend on the number of predefined slots, and allow intuitive evaluation .

Copied to clipboard

Haoning Zhang, Junwei Bao, Haipeng Sun, Youzheng Wu, Wenye Li, Shuguang Cui, Xiaodong He

Towards Fair Evaluation of Dialogue State Tracking by Flexible Incorporation of Turn-level Performances (2022.acl-short)

Challenge:	Experimental results show that MoNET outperforms previous DST methods in alleviating state momentum issues and improving the anti-noise ability.
Approach:	They propose to use previous state of each turn in training data as input to learn to predict current state.
Outcome:	The proposed model outperforms existing methods on multiWOZ datasets and shows that it can update and correct slot values and improve anti-noise ability.

Copied to clipboard

Suvodip Dey, Ramamohan Kummara, Maunendra Desarkar

CSS: Combining Self-training and Self-supervised Learning for Few-shot Dialogue State Tracking (2022.aacl-short)

Challenge:	Dialogue State Tracking (DST) is a task-oriented conversational agent that keeps track of key information exchanged during a conversation.
Approach:	They propose a new evaluation metric called Flexible Goal Accuracy to address shortcomings of JGA.
Outcome:	The proposed metric improves on existing metrics and improves performance of turn-level and non-cumulative belief state models.

Copied to clipboard

Haoning Zhang, Junwei Bao, Haipeng Sun, Huaishao Luo, Wenye Li, Shuguang Cui

Slot Dependency Modeling for Zero-Shot Cross-Domain Dialogue State Tracking (2022.coling-1)

Challenge:	Existing few-shot dialogue state tracking (DST) methods transfer knowledge from labeled data into DST, but collecting large amount of labeles is laborious.
Approach:	They propose a few-shot dialogue state tracking framework that integrates self-training and self-supervised learning methods into the framework.
Outcome:	The proposed framework achieves competitive performance in several few-shot scenarios.

Copied to clipboard

Qingyue Wang, Yanan Cao, Piji Li, Yanhe Fu, Zheng Lin, Li Guo

Towards LLM-driven Dialogue State Tracking (2023.emnlp-main)

Challenge:	Existing zero-shot learning methods ignore slot dependencies in a multidomain dialogue . experimental results show the effectiveness of our proposed method over existing state-of-art generation methods .
Approach:	They propose to use slot prompts combination, slot values demonstration and slot constraint object to model slot-slot dependency, slot-value dependency and slot-context dependency respectively.
Outcome:	The proposed method outperforms state-of-the-art methods under zero-shot/few-shot settings.

Copied to clipboard

Yujie Feng, Zexin Lu, Bo Liu, Liming Zhan, Xiao-Ming Wu

Efficient Dialogue State Tracking by Selectively Overwriting Memory (2020.acl-main)

Challenge:	emergence of large language models (LLMs) such as GPT3 and ChatGPT has sparked considerable interest in assessing their efficacy across diverse applications.
Approach:	They present a framework for a domain-slot instruction tuning method that allows LDST to achieve performance on par with ChatGPT.
Outcome:	The proposed framework performs better in zero-shot and few-shot settings than previous SOTA methods.

Copied to clipboard

Sungdong Kim, Sohee Yang, Gyuwan Kim, Sang-Woo Lee

Correctable-DST: Mitigating Historical Context Mismatch between Training and Inference for Improved Dialogue State Tracking (2022.emnlp-main)

Challenge:	Recent work in dialogue state tracking (DST) is inefficient in that it predicts dialogue state at every turn from scratch.
Approach:	They propose a method that selectively overwrites memory for dialogue state tracking by predicting dialogue state on each memory slot and overwriting it with new values.
Outcome:	The proposed model achieves state-of-the-art joint goal accuracy with 51.72% in MultiWOZ 2.0 and 53.01% in MultiWoz 2.1 in an open vocabulary-based DST setting.

Copied to clipboard

Hongyan Xie, Haoxiang Su, Shuangyong Song, Hao Huang, Bo Zou, Kun Deng, Jianghua Lin, Zhihui Zhang, Xiaodong He

Improving Dialogue State Tracking by Discerning the Relevant Context (N19-1)

Challenge:	Existing dialogue state tracking approaches predict the dialogue state of a target turn sequentially based on the ground-truth previous dialogue state.
Approach:	They propose a method that predicts dialogue state sequentially based on previous dialogue state . they propose generating a previously “predicted” dialogue state using ground-truth previous dialogue states .
Outcome:	The proposed method achieves 67.51%, 68.24%, 70.30%, 71.38%, and 81.27% joint goal accuracy on MultiWOZ 2.0-2.4 datasets.

Copied to clipboard

Sanuj Sharma, Prafulla Kumar Choubey, Ruihong Huang

Zero and Few-Shot Localization of Task-Oriented Dialogue Agents with a Distilled Representation (2023.eacl-main)

Challenge:	Dialog state tracking (DST) is used to estimate user's goals and requests in order to plan next action and respond accordingly.
Approach:	They propose a framework that uses the current user utterance and the most recent system utterant to determine the relevance of a system . Specifically, they use the current and most recent user . and system adverbs to determine relevance.
Outcome:	The proposed framework improves goal accuracy by 2.75% and 2.36% on WoZ 2.0 and Multi-WoZ restaurant domain datasets over the previous state-of-the-art GLAD model.

Copied to clipboard

Mehrad Moradshahi, Sina Semnani, Monica Lam

Multilinguality

Challenge:	Existing low-cost approaches to build a high-quality functioning dialogue agent are limited to a few widely-spoken languages.
Approach:	They propose automatic methods that use ToD training data to build a functioning agent in another language . they compare the method to existing methods that only use a small training set .
Outcome:	The proposed method improves the state-of-the-art in Chinese to English transfer using zero-shot data compared to existing full-shot methods . the proposed method achieves 46.7% and 22.0% in task success rate and dialogue success rate, respectively.

Robust Dialogue State Tracking with Weak Supervision and Sparse Data (2022.tacl-1)

Copied to clipboard

Michael Heck, Nurul Lubis, Carel van Niekerk, Shutong Feng, Christian Geishauser, Hsien-Chin Lin, Milica Gašić

Efficient Context and Schema Fusion Networks for Multi-Domain Dialogue State Tracking (2020.findings-emnlp)

Challenge:	Generalizing dialogue state tracking (DST) to new data and domains is especially challenging due to the strong reliance on abundant and fine-grained supervision during training.
Approach:	They propose a training strategy to build extractive DST models without the need for fine-grained manual span labels.
Outcome:	The proposed model improves robustness against sample sparsity, new concepts, and topics, leading to state-of-the-art performance on a range of benchmarks.

Copied to clipboard

Su Zhu, Jieyu Li, Lu Chen, Kai Yu

TaSL: Continual Dialog State Tracking via Task Skill Localization and Consolidation (2024.acl-long)

Challenge:	Existing methods to track dialogue state are limited due to data sparsity and long dialogues.
Approach:	They propose to use the previous dialogue state and current dialogue utterance as input for DST.
Outcome:	The proposed approach outperforms existing methods and improves existing ones.

Copied to clipboard

Yujie Feng, Xu Chu, Yongxin Xu, Guangyuan Shi, Bo Liu, Xiao-Ming Wu

DS-TOD: Efficient Domain Specialization for Task-Oriented Dialog (2022.findings-acl)

Challenge:	Current methods for Continual Dialogue State Tracking (DST) struggle with catastrophic forgetting and knowledge transfer between tasks.
Approach:	They propose a framework for task skill localization and consolidation that enables effective knowledge transfer without relying on memory replay.
Outcome:	The proposed framework shows a 7.6% increase in Avg. JGA and 11% rise in BWT metrics over existing state-of-the-art methods.

Copied to clipboard

Chia-Chien Hung, Anne Lauscher, Simone Ponzetto, Goran Glavaš

XQA-DST: Multi-Domain and Multi-Lingual Dialogue State Tracking (2023.findings-eacl)

Challenge:	Recent work shows that self-supervised dialog-specific pretraining on large conversational datasets yields substantial gains over traditional language modeling (LM) pretraining.
Approach:	They propose a resource-efficient and modular domain specialization by means of domain adapters in which domain knowledge is encoded.
Outcome:	The proposed framework extracts domain-specific terms and then uses them to build DomainCC and DomainReddit resources based on masked language modeling and response selection objectives.

Copied to clipboard

Han Zhou, Ignacio Iacobacci, Pasquale Minervini

MetaASSIST: Robust Dialogue State Tracking with Meta Learning (2022.emnlp-main)

Challenge:	Existing methods for capturing dialogue data are expensive and limited in their application.
Approach:	They propose a domain-agnostic extractive question answering approach with shared weights across domains to disentangle complex domain information in ToDs.
Outcome:	The proposed model can efficiently leverage domain-agnostic QA datasets while being domain-scalable and open vocabulary in DST.

Copied to clipboard

Fanghua Ye, Xi Wang, Jie Huang, Shenghui Li, Samuel Stern, Emine Yilmaz

Value type: the bridge to a better DST model (2023.findings-acl)

Challenge:	Existing dialogue datasets contain lots of noise in their state annotations.
Approach:	They propose a framework to train robust dialogue state tracking models by combining pseudo and vanilla labels by a common weighting parameter.
Outcome:	The proposed framework achieves state-of-the-art accuracy of 80.10% on multiWOZ 2.4.

Copied to clipboard

Gao Qixiang, Mingyang Sun, Yutao Mou, Chen Zeng, Weiran Xu

Named Entity Recognition

Challenge:	Value type of the slots can provide lots of useful information for DST tasks. however, it has been ignored in most previous works.
Approach:	They propose a new framework for DST task based on slot value type . they propose to extract the type of token from each turn and train a Ner model to extract corresponding type-entity from each conversation according to the token.
Outcome:	The proposed framework is effective on two multi-domain task-oriented conversation datasets.

ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity? (2023.acl-short)

Copied to clipboard

Michael Heck, Nurul Lubis, Benjamin Ruppik, Renato Vukovic, Shutong Feng, Christian Geishauser, Hsien-chin Lin, Carel van Niekerk, Milica Gasic

From Schema to State: Zero-Shot Scheme-Only Dialogue State Tracking via Diverse Synthetic Dialogue and Step-by-Step Distillation (2025.emnlp-main)

Challenge:	Recent research on dialog state tracking (DST) focuses on methods that allow few- and zero-shot transfer to new domains or schemas.
Approach:	They propose to use schema descriptions to facilitate zero-shot transfer to new domains . they argue that general purpose language models lack the ability to replace specialized systems .
Outcome:	The proposed method achieves state-of-the-art in zero-shot DST with in-context learning capabilities.

Copied to clipboard

Huan Xu, Zequn Li, Wen Tang, Jian Jun Zhang

Zero-shot Generalization in Dialog State Tracking through Generative Question Answering (2021.eacl-main)

Challenge:	Existing research classifies zero-shot, scheme-only DST into two main types: the cross-domain scenario and the zero-schemaonly setting.
Approach:	They propose a zero-shot, scheme-only approach that generates synthetic dialogues that balance diversity with schema alignment and distills knowledge from a large language model into a smaller model.
Outcome:	The proposed approach achieves state-of-the-art performance under zero-shot, scheme-only situation and generalizes effectively to few-shot scenarios.

Copied to clipboard

Shuyang Li, Jin Cao, Mukund Sridhar, Henghui Zhu, Shang-Wen Li, Wael Hamza, Julian McAuley

GCDST: A Graph-based and Copy-augmented Multi-domain Dialogue State Tracking (2020.findings-emnlp)

Challenge:	Existing methods for Dialog State Tracking do not generalize well to new domains and unseen slots.
Approach:	They propose an ontology-free framework that queries for unseen constraints and slots in multi-domain task-oriented dialogs using a conditional language model pre-trained on substantive English sentences.
Outcome:	The proposed framework improves goal accuracy in zero-shot domain adaptation settings by up to 9% over the previous state-of-the-art on the MultiWOZ 2.1 dataset.

Copied to clipboard

Peng Wu, Bowei Zou, Ridong Jiang, AiTi Aw

More Robust Schema-Guided Dialogue State Tracking via Tree-Based Paraphrase Ranking (2023.findings-eacl)

Challenge:	Existing approaches to training DST on a single domain ignore information across domains.
Approach:	They construct a dialogue state graph to transfer structured features among related domain-slot pairs across domains and encode the graph information of dialogue states by graph convolutional networks.
Outcome:	The proposed model improves the performance of the multi-domain DST baseline with the absolute joint accuracy of 2.0% and 1.0% on the MultiWOZ 2.0 and 2.1 dialogue datasets.

Copied to clipboard

Alexandru Coca, Bo-Hsiang Tseng, Weizhe Lin, Bill Byrne

Preview, Attend and Review: Schema-Aware Curriculum Learning for Multi-Domain Dialogue State Tracking (2021.acl-short)

Challenge:	Pre-trained language models (PLMs) can track only slots drawn from a database or domain ontology.
Approach:	They propose a framework for generating synthetic schemas which uses tree-based ranking to optimise lexical diversity and semantic faithfulness.
Outcome:	The proposed framework improves the generalisation of strong baselines by augmenting training data with prompts generated by the framework.

Copied to clipboard

Yinpei Dai, Hangyu Li, Yongbin Li, Jian Sun, Fei Huang, Luo Si, Xiaodan Zhu

Divide, Conquer, and Combine: Mixture of Semantic-Independent Experts for Zero-Shot Dialogue State Tracking (2023.acl-long)

Challenge:	Existing dialog state tracking models neglect rich structural information in a dataset.
Approach:	They propose to use curriculum learning to leverage dialog state tracking data . they propose a model-agnostic framework that pre-trains a DST model with schema information .
Outcome:	The proposed framework improves performance over a transformer-based and RNN-based model on WOZ2.0 and MultiWOZ2.1.

Copied to clipboard

Qingyue Wang, Liang Ding, Yanan Cao, Yibing Zhan, Zheng Lin, Shi Wang, Dacheng Tao, Li Guo

Stabilized In-Context Learning with Pre-trained Language Models for Few Shot Dialogue State Tracking (2023.findings-eacl)

Challenge:	Existing methods to enhance the zeroshot generalization of DST fail to effectively decouple semantics of samples, limiting the zero-shot performance of the system.
Approach:	They propose a new learning schema that explicitly disentangles the semantics of seen data and leverages the performance and robustness with the mixture-of-experts mechanism.
Outcome:	The proposed model achieves state-of-the-art on multiWOZ2.1 with 10M trainable parameters and is robust to the mixture-of experts mechanism.

Copied to clipboard

Derek Chen, Kun Qian, Zhou Yu

SynthDST: Synthetic Data is All You Need for Few-Shot Dialog State Tracking (2024.eacl-long)

Challenge:	Prompt-based methods with large pre-trained language models have shown impressive unaided performance across many NLP tasks.
Approach:	They propose a meta-learning scheme to stabilize the ability of the model to perform well under various prompts and introduce a saliency model to limit dialogue text length.
Outcome:	The proposed model improves on large pre-trained language models with labeled in-context exemplars and can be used to generate more exemplar queries.

Copied to clipboard

Atharva Kulkarni, Bo-Hsiang Tseng, Joel Moniz, Dhivya Piraviperumal, Hong Yu, Shruti Bhargava

Unsupervised Speech-text word-level alignment with Dynamic Programming (2025.findings-naacl)

Challenge:	In-context learning with Large Language Models (LLMs) is a promising avenue of research in Dialog State Tracking (DST).
Approach:	They propose a data generation framework tailored for Dialog State Tracking that uses large language models to synthesize natural, coherent, and free-flowing dialogues with DST annotations.
Outcome:	The proposed framework improves joint goal accuracy by 4-5% over the zero-shot baseline on MultiWOZ 2.1 and 2.4.

Copied to clipboard

Tianshu Yu, Zihan Gong, Minghuan Tan, Guhong Chen, Min Yang

Multimodality

Challenge:	Word-level alignment in speech-text pretraining models is limited by word-level annotated data . authors propose an iterative training method for USDP that reduces the dependency on scarce annotation resources.
Approach:	They propose an Unsupervised Speech-text word-level alignment with Dynamic Programming (USDP) this method uses Dynamic programming principles to iteratively refine temporal alignment predictions .
Outcome:	The proposed method significantly improves on speech-text pretraining tasks compared to existing methods.

N-Shot Learning for Augmenting Task-Oriented Dialogue State Tracking (2022.findings-acl)

Copied to clipboard

Ibrahim Aksu, Zhengyuan Liu, Min-Yen Kan, Nancy Chen

An End-to-end Approach for Handling Unknown Slot Values in Dialogue State Tracking (P18-1)

Challenge:	augmentation of task-oriented dialogues has followed standard methods for plain-text despite its richly annotated structure.
Approach:	They propose an augmentation framework that utilizes belief state annotations to match turns from various dialogues and form new synthetic dialogues in a bottom-up manner.
Outcome:	The proposed framework performs better on seen values and more robust to unseen values on n-shot training scenarios.

Copied to clipboard

Puyang Xu, Qi Hu

A Sequence-to-Sequence Approach to Dialogue State Tracking (2021.acl-long)

Challenge:	a dialogue state tracker is a core component in most of today's spoken dialogue systems . slot-filling dialogues are composed of a predefined set of slots that need to be filled through the conversation .
Approach:	They propose an E2E architecture that extracts unknown slot values while still achieving state-of-the-art accuracy on the standard DSTC2 benchmark.
Outcome:	The proposed architecture achieves state-of-the-art accuracy on the DSTC2 benchmark while retaining predefined slot values.

Copied to clipboard

Yue Feng, Yang Wang, Hang Li

Neural Dialogue State Tracking with Temporally Expressive Networks (2020.findings-emnlp)

Challenge:	Existing methods for dialogue state tracking are still challenging, but they are improving . a new approach to dialogue state monitoring is proposed, called Seq2Seq-DU .
Approach:	They propose a new dialogue state tracking module that formalizes DST as a sequence-to-sequence problem.
Outcome:	The proposed method outperforms existing methods on benchmark datasets in different settings.

Copied to clipboard

Junfan Chen, Richong Zhang, Yongyi Mao, Jie Xu

Improving Dialogue State Tracking with Turn-based Loss Function and Sequential Data Augmentation (2021.findings-emnlp)

Challenge:	Existing models ignore temporal feature dependencies across dialogue turns or fail to explicitly model temporal state dependencies in a dialogue.
Approach:	They propose to combine temporal feature dependencies in spoken dialogues by using recurrent networks and probabilistic graphical models.
Outcome:	The proposed model improves turn-level-state prediction and state aggregation on standard datasets.

Copied to clipboard

Jarana Manotumruksa, Jeff Dalton, Edgar Meij, Emine Yilmaz

Few Shot Dialogue State Tracking using Meta-learning (2021.eacl-main)

Challenge:	Existing models rely on a traditional cross-entropy loss function during training, which may not be optimal for improving the joint goal accuracy.
Approach:	They propose a Turn-based Loss Function that penalises the model if it inaccurately predicts a slot value at the early turns more so than in later turns to improve joint goal accuracy.
Outcome:	The proposed techniques improve the state-of-the-art model by approximately 7-8% relative reduction in error and achieve a new state- of-the art joint goal accuracy with 59.50 and 54.90 on MultiWOZ2.1 and MultiWOz2.2, respectively.

Copied to clipboard

Saket Dingliwal, Shuyang Gao, Sanchit Agarwal, Chien-Wei Lin, Tagyoung Chung, Dilek Hakkani-Tur

Exploiting domain-slot related keywords description for Few-Shot Cross-Domain Dialogue State Tracking (2022.emnlp-main)

Challenge:	Existing methods for transferring knowledge from resource-rich domains to unknown domains are data hungry . a meta-learning algorithm is proposed to solve the problem of zero/few-shot DST .
Approach:	They propose a meta-learner for the problem of zero/few-shot DST . they propose to agnostically train any existing chatbot system to improve its performance .
Outcome:	The proposed meta-learner improves on baseline in a low-data setting.

Copied to clipboard

Gao Qixiang, Guanting Dong, Yutao Mou, Liwen Wang, Chen Zeng, Daichi Guo, Mingyang Sun, Weiran Xu

BREAK: Breaking the Dialogue State Tracking Barrier with Beam Search and Re-ranking (2023.acl-long)

Challenge:	Existing frameworks for dialogue state tracking with domain-slot-value labels are expensive . current models are limited due to high cost of data annotation and lack of data in some domains .
Approach:	They propose a framework based on domain-slot related description to tackle the challenge of few-shot cross-domain DST.
Outcome:	The proposed framework outperforms existing methods on MultiWOZ and gains strong slot accuracy compared to existing models.

Copied to clipboard

Seungpil Won, Heeyoung Kwak, Joongbo Shin, Janghoon Han, Kyomin Jung

Domain-Lifelong Learning for Dialogue State Tracking via Knowledge Preservation Networks (2021.emnlp-main)

Challenge:	Existing methods for dialogue state tracking still have a JGA of 60% on MultiWOZ 2.1 . break framework provides a simple yet effective way to generate dialogue state candidates .
Approach:	They propose a framework that generates k-best dialogue state candidates with beam search and re-ranks them to select the correct dialogue state.
Outcome:	The proposed framework pushes the joint goal accuracy to 80-90% on MultiWOZ 2.1-2.4.

Copied to clipboard

Qingbin Liu, Pengfei Cao, Cao Liu, Jiansong Chen, Xunliang Cai, Fan Yang, Shizhu He, Kang Liu, Jun Zhao

Recursive Template-based Frame Generation for Task Oriented Dialog (2020.acl-main)

Challenge:	Existing offline DST models require a fixed dataset to train . Existing domain-lifelong learning methods are impractical in real-world applications .
Approach:	They propose a domain-lifelong learning method to continuously train a DST model on new data to learn incessantly emerging new domains while avoiding catastrophically forgetting old learned domains.
Outcome:	The proposed method outperforms state-of-the-art lifelong learning methods by 4.25% and 8.27% on the MultiWOZ and the SGD benchmarks.

Copied to clipboard

Rashmi Gangadharaiah, Balakrishnan Narayanaswamy

UNO-DST: Leveraging Unlabelled Data in Zero-Shot Dialogue State Tracking (2024.findings-naacl)

Challenge:	The input of an NLU component is a semantic frame that captures the intent and slot-labels provided by the user.
Approach:	They propose a recursive, hierarchical representation that captures the intent and slot-labels provided by the user and extend local tree-based loss functions with terms that provide global supervision.
Outcome:	The proposed representation improves on the widely used ATIS dataset and significantly improves the performance of the proposed framework.

Copied to clipboard

Chuang Li, Yan Zhang, Min-Yen Kan, Haizhou Li

In-Context Learning for Few-Shot Dialogue State Tracking (2022.findings-emnlp)

Challenge:	Existing methods for zero-shot dialogue state tracking (DST) ignore unlabelled data in the target domain.
Approach:	They propose to transform zero-shot dialogue state tracking into few-shot DST by utilising unlabelled data via joint and self-training methods.
Outcome:	The proposed method improves joint goal accuracy by 8% on general language models in zero-shot scenarios, and can be used in many domains.

Copied to clipboard

Yushi Hu, Chia-Hsuan Lee, Tianbao Xie, Tao Yu, Noah A. Smith, Mari Ostendorf

Comprehensive Study: How the Context Information of Different Granularity Affects Dialogue State Tracking? (2021.acl-long)

Challenge:	Existing methods for zero-shot and few-shot learning dialogue state tracking are hard and expensive.
Approach:	They propose an in-context learning framework for zero-shot and few-shot learning dialogue state tracking (DST) a large pretrained language model takes a test instance and a few exemplars as input and directly decodes the dialogue state .
Outcome:	The proposed framework outperforms state-of-the-art models in few-shot settings . it is flexible and scalable, and requires less data to adapt to new domains and scenarios .

Copied to clipboard

Puhai Yang, Heyan Huang, Xian-Ling Mao

Scalable and Accurate Dialogue State Tracking via Hierarchical Sequence Generation (D19-1)

Challenge:	Dialogue state tracking (DST) plays a key role in task-oriented dialogue systems to monitor the user’s goal.
Approach:	They propose to use scratch-based and previous-based strategies to track dialogue state . they explore how different granularities affect dialogue state tracking .
Outcome:	The scratch-based strategy obtains each slot value by inquiring all the dialogue history, while the previous-based method is not very useful for long-dependency dialogue state tracking.

Copied to clipboard

Liliang Ren, Jianmo Ni, Julian McAuley

ASSIST: Towards Label Noise-Robust Dialogue State Tracking (2022.findings-acl)

Challenge:	Existing approaches to dialogue state tracking rely on pre-defined ontologies . however, these methods suffer from computational complexity that increases proportionally to the number of pre-determined slots.
Approach:	They propose a model that generates a sequence of belief states without the pre-defined ontology list.
Outcome:	The proposed model scales easily with the increasing number of pre-defined slots and domains and reaches the state-of-the-art performance on the multi-domain and single domain dialogue state tracking datasets.

Copied to clipboard

Fanghua Ye, Yue Feng, Emine Yilmaz

Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systems (2020.findings-emnlp)

Challenge:	Existing versions of MultiWOZ 2.0 have been published, but there are still lots of noisy labels in the training set.
Approach:	They propose a framework to train dialogue state tracking models from noisy labels instead of improving annotation quality further by using auxiliary models.
Outcome:	The proposed framework improves the goal accuracy of DST models by 28.16% on MultiWOZ 2.0 and 8.41% on MultiWoz 2.4, compared to using only the vanilla noisy labels.

Copied to clipboard

Andrea Madotto, Samuel Cahyawijaya, Genta Indra Winata, Yan Xu, Zihan Liu, Zhaojiang Lin, Pascale Fung

Enhancing Task-oriented Dialogue Systems with Generative Post-processing Networks (2023.emnlp-main)

Challenge:	End-to-end systems rely on dialogue state tracking and annotations to fulfill user requests . modularized systems require multiple steps, including a direct interaction with the KB .
Approach:	They propose a method to embed the KB directly into the model parameters . they evaluate five task-oriented dialogue datasets with small, medium, and large KBs .
Outcome:	The proposed model can embed the KB directly into the model parameters without any DST or template responses, nor the kb as input.

Copied to clipboard

Atsumoto Ohashi, Ryuichiro Higashinaka

EDC: Effective and Efficient Dialog Comprehension For Dialog State Tracking (2024.naacl-long)

Challenge:	Recent work proposes a method to optimize pipelined dialogue systems by fine-tuning modules directly.
Approach:	They propose a new post-processing component for natural language generation (NLG) they use dialogue act contribution to evaluate contribution of GenPPN-generated utterances .
Outcome:	The proposed method improves the performance of task-oriented dialogue systems by modifying arbitrary modules including non-differentiable ones.

Copied to clipboard

Qifan Lu, Bhaskar Ramasubramanian, Radha Poovendran

Continual Dialogue State Tracking via Example-Guided Question Answering (2023.emnlp-main)

Challenge:	Existing methods for dialog state tracking face trade-offs between accuracy and efficiency . effective and efficient dialog comprehension (EDC) predicts domains, slot names and slot values of dialog state step-by-step for better accuracy .
Approach:	They propose an alternative method that leverages the tree structure of the dialog state.
Outcome:	The proposed approach achieves state-of-the-art JGA accuracy and is more efficient than previous models.

Copied to clipboard

Hyundong Cho, Andrea Madotto, Zhaojiang Lin, Khyathi Chandu, Satwik Kottur, Jing Xu, Jonathan May, Chinnadhurai Sankar

LUNA: Learning Slot-Turn Alignment for Dialogue State Tracking (2022.naacl-main)

Challenge:	Dialogue systems are frequently updated to accommodate new services, but naively updating them by continually training with data for new services causes catastrophic forgetting.
Approach:	They propose to reformulate dialogue state tracking (DST) as a bundle of example-guided question answering tasks to minimize the task shift between services.
Outcome:	The proposed model achieves state-of-the-art performance on DST continual learning metrics without relying on any complex regularization or parameter expansion methods.

Copied to clipboard

Yifan Wang, Jing Zhao, Junwei Bao, Chaoqun Duan, Youzheng Wu, Xiaodong He

Slot Attention with Value Normalization for Multi-Domain Dialogue State Tracking (2020.emnlp-main)

Challenge:	Existing methods exploit the utterances of all dialogue turns to assign value to slots . this can lead to suboptimal results due to information introduced from irrelevant utterrances .
Approach:	They propose a SLot-TUrN Alignment enhanced approach to assign slot value . they explicitly align each slot with its most relevant utterance and then predict the corresponding value based on this aligned utteration.
Outcome:	The proposed approach achieves state-of-the-art on three multi-domain task-oriented dialogue datasets.

Copied to clipboard

Yexiang Wang, Yi Guo, Siqi Zhu

Prompter: Zero-shot Adaptive Prefixes for Dialogue State Tracking Domain Adaptation (2023.acl-long)

Challenge:	Existing dialogue state tracking approaches rely on ontology already defined, where all slots and their possible values are given.
Approach:	They propose a new architecture to exploit domain ontology by using Slot Attention and Value Normalization . they supplement the annotation of supporting span for MultiWOZ 2.1, which is the shortest span in utterances to support the labeled value.
Outcome:	The proposed architecture exploits ontology and can convert supporting spans to values.

Copied to clipboard

Ibrahim Taha Aksu, Min-Yen Kan, Nancy Chen

Dialogue Summaries as Dialogue States (DS2), Template-Guided Summarization for Few-shot Dialogue State Tracking (2022.findings-acl)

Challenge:	Parameter-Efficient Transfer Learning (PETL) has the potential to address this problem, but it has yet to be applied to the zero-shot domain adaptation.
Approach:	They propose to use descriptions of target domain slots to generate dynamic prefixes that are concatenated to the key and values at each layer’s self-attention mechanism.
Outcome:	The proposed method outperforms previous methods on the MultiWOZ and SGD benchmarks.

Copied to clipboard

Jamin Shin, Hangyeol Yu, Hyeongdon Moon, Andrea Madotto, Juneyoung Park

Multi-Domain Dialogue State Tracking with Disentangled Domain-Slot Attention (2023.findings-acl)

Challenge:	Annotating task-oriented dialogues is notorious for the expensive and difficult data collection process.
Approach:	They propose to reformulate dialogue state tracking as a dialogue summarization problem by using synthetic dialogue summaries generated by a set of rules.
Outcome:	The proposed method outperforms previous studies on few-shot dialogue state tracking in MultiWoZ 2.0 and 2.1 in cross-domain and multi-domain settings.

Copied to clipboard

Longfei Yang, Jiyi Li, Sheng Li, Takahiro Shinozaki

DiSTRICT: Dialogue State Tracking with Retriever Driven In-Context Tuning (2023.emnlp-main)

Challenge:	Multi-domain dialogue state tracking is a challenge for task-oriented dialogue systems . domains and slots are aggregated into a single query to generate domain-slot specific representations .
Approach:	They propose to disentangle domain-slot attention for multi-domain dialogue state tracking by separating query about domains and slots from the attention component.
Outcome:	The proposed approach outperforms the standard multi-head attention with aggregated domain-slot query.

Copied to clipboard

Praveen Venkateswaran, Evelyn Duesterwald, Vatche Isahagian

Zero-Shot Cross-Domain Dialogue State Tracking via Dual Low-Rank Adaptation (2024.acl-long)

Challenge:	Existing approaches to task-oriented conversation system DST use hand-crafted templates and additional slot information to fine-tune and prompt large pre-trained language models and elicit slot values from the dialogue context.
Approach:	They propose a generalizable in-context tuning approach that retrieves highly relevant training examples for a given dialogue to fine-tune the model without any hand-crafted templates.
Outcome:	Experiments with the MultiWOZ benchmark datasets show that DiSTRICT outperforms existing approaches in various zero-shot and few-shot settings using a much smaller model.

Copied to clipboard

Xiang Luo, Zhiwen Tang, Jin Wang, Xuejie Zhang

Common Ground Tracking in Multimodal Dialogue (2024.lrec-main)

Challenge:	Existing approaches to zero-shot dialogue state tracking (DST) involve embedding prompts into language models, but these methods have inherent limitations.
Approach:	They propose a plug-and-play architecture designed for zero-shot dialogue state tracking (DST) dual low-rank adaptation targets dialogue context processing and prompt optimization without incurring additional inference latency.
Outcome:	The proposed architecture outperforms baseline methods on multi-domain datasets and the MultiWOZ dataset.

Copied to clipboard

Ibrahim Khalil Khebour, Kenneth Lai, Mariah Bradford, Yifan Zhu, Richard A. Brutti, Christopher Tam, Jingxuan Tu, Benjamin A. Ibarra, Nathaniel Blanchard, Nikhil Krishnaswamy, James Pustejovsky

Diverse Retrieval-Augmented In-Context Learning for Dialogue State Tracking (2023.findings-acl)

Challenge:	In dialogue modeling, there is considerable attention on “dialogue state tracking” (DST) but “common ground tracking” identifies the shared belief space held by all participants in a task-oriented dialogue: the task-relevant propositions all participants accept as true.
Approach:	They propose a method for automatically identifying the current set of shared beliefs and ”questions under discussion” of a group with a shared goal.
Outcome:	The proposed method predicts moves toward building common ground relative to ground truth in a multimodal interaction with an AI.

Copied to clipboard

Brendan King, Jeffrey Flanigan

A Zero-Shot Open-Vocabulary Pipeline for Dialogue Understanding (2025.naacl-long)

Challenge:	Recent work has demonstrated that in-context learning for dialogue state tracking outperforms training methods in the few-shot setting.
Approach:	They propose a method for in-context learning for dialogue state tracking that takes into account probabilities of competing surface forms and produces a more accurate dialogue state prediction.
Outcome:	The proposed method outperforms trained methods in the few-shot setting and requires little data and zero parameter updates.

Copied to clipboard

Abdulfattah Safa, Gözde Gül Şahin

Know Thy Strengths: Comprehensive Dialogue State Tracking Diagnostics (2022.findings-emnlp)

Challenge:	Existing approaches to DST are limited by their computational resources or lack flexibility to adapt to new slots.
Approach:	They propose a system that integrates domain classification and DST in a single pipeline and uses self-refining prompts to adapt dynamically.
Outcome:	The proposed system improves on existing methods on multiWOZ datasets and provides 20% better Joint Goal Accuracy (JGA) over existing methods with 90% fewer requests to the LLM API.

Copied to clipboard

Hyundong Cho, Chinnadhurai Sankar, Christopher Lin, Kaushik Sadagopan, Shahin Shayandeh, Asli Celikyilmaz, Jonathan May, Ahmad Beirami

Improving Limited Labeled Dialogue State Tracking with Self-Supervision (2020.findings-emnlp)

Challenge:	Recent studies have revealed the vulnerability of dialogue state tracking models to distributional shifts, resulting in poor performance.
Approach:	They present a toolkit for standardized and comprehensive dialogue state tracking diagnoses that provides a richer summary of strengths and weaknesses.
Outcome:	The proposed toolkit shows that different classes of DST models have clear strengths and weaknesses, while generation models are more promising for handling language variety and span-based classification models are robust to unseen entities.

Copied to clipboard

Chien-Sheng Wu, Steven C.H. Hoi, Caiming Xiong

Continual Dialogue State Tracking via Reason-of-Select Distillation (2024.findings-acl)

Challenge:	Existing dialogue state tracking models require plenty of labeled data, but collecting labels is expensive.
Approach:	They propose to use only 1% labeled data to train dialogue state tracking models . they encourage a model to have consistent latent distributions given a perturbed input .
Outcome:	The proposed self-supervised signals improve goal accuracy by 8.95% when only 1% labeled data is used on the MultiWOZ dataset.

Copied to clipboard

Yujie Feng, Bo Liu, Xiaoyu Dong, Zexin Lu, Li-Ming Zhan, Xiao-Ming Wu, Albert Lam

Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue StateTracking (2021.naacl-main)

Challenge:	Existing research on dialogue systems has focused on domain-specific offline systems lacking adaptation abilities.
Approach:	They propose a Reason-of-Select distillation method that enhances smaller models with a novel "meta-reasoning" capability.
Outcome:	Experiments show that the proposed method significantly improves the performance and generalization capabilities of existing models.

Copied to clipboard

Zhaojiang Lin, Bing Liu, Seungwhan Moon, Paul Crook, Zhenpeng Zhou, Zhiguang Wang, Zhou Yu, Andrea Madotto, Eunjoon Cho, Rajen Subba

Task-Optimized Adapters for an End-to-End Task-Oriented Dialogue System (2023.findings-acl)

Challenge:	Existing models for zero-shot cross-domain dialogue state tracking require in-domain data to model a new domain.
Approach:	They propose a slot descriptions enhanced generative approach for zero-shot cross-domain DST by encoding a dialogue context and a slots with a pre-trained encoder and generating slot value in auto-regressive manner.
Outcome:	The proposed model significantly improves state-of-the-art results in zero-shot cross-domain setting.

Copied to clipboard

Namo Bang, Jeehyun Lee, Myoung-Wan Koo

Large Language Models as Zero-shot Dialogue State Tracker through Function Calling (2024.acl-long)

Challenge:	Recent work on end-to-end dialogue models with pre-trained dialogue corpora shows promising performance in the conversational system.
Approach:	They propose an end-to-end TOD system with task-optimized adapters which learn independently per task adding only small number of parameters after fixed layers of pre-trained network.
Outcome:	The proposed system achieves state-of-the-art performance on the MultiWOZ benchmark compared to existing models.

Copied to clipboard

Zekun Li, Zhiyu Chen, Mike Ross, Patrick Huber, Seungwhan Moon, Zhaojiang Lin, Xin Dong, Adithya Sagar, Xifeng Yan, Paul Crook

Rewarding What Matters: Step-by-Step Reinforcement Learning for Task-Oriented Dialogue (2024.findings-emnlp)

Challenge:	Large language models (LLMs) are increasingly prevalent in conversational systems due to their advanced understanding and generative capabilities in general contexts.
Approach:	They propose a method for solving dialogue state tracking (DST) with large language models through function calling.
Outcome:	The proposed approach improves zero-shot DST, allowing adaptation to diverse domains without extensive data collection or model tuning.

Copied to clipboard

Huifang Du, Shuqin Li, Minghao Wu, Xuejing Feng, Yuan-Fang Li, Haofen Wang

Enhancing Dialogue State Tracking Models through LLM-backed User-Agents Simulation (2024.acl-long)

Challenge:	Existing RL methods focus on generation tasks while neglecting dialogue state tracking (DST) for understanding.
Approach:	They propose a method that integrates RL into both understanding and generation tasks by introducing step-by-step rewards throughout the token generation.
Outcome:	The proposed approach achieves state-of-the-art results on three widely used datasets.

Copied to clipboard

Cheng Niu, Xingguang Wang, Xuxin Cheng, Juntong Song, Tong Zhang

Turn-Level Active Learning for Dialogue State Tracking (2023.emnlp-main)

Challenge:	Experimental results show that the model can be used to generate dialogues in new domains quickly.
Approach:	They propose to use LLMs to generate dialogue data to reduce dialogue collection and annotation costs.
Outcome:	The proposed model performs better than the baseline model trained on real data.

Copied to clipboard

Zihan Zhang, Meng Fang, Fanghua Ye, Ling Chen, Mohammad-Reza Namazi-Rad

Decoder-only Streaming Transformer for Simultaneous Translation (2024.acl-long)

Challenge:	Existing approaches to annotate dialogues require supervised training, which requires human workers to manually annotates dialogues.
Approach:	They propose a turn-level active learning framework to actively select dialogue turns to annotate . their approach can achieve comparable performance to traditional training approaches .
Outcome:	The proposed model achieves comparable performance to existing training approaches with significantly less annotated data.

Copied to clipboard

Shoutao Guo, Shaolei Zhang, Yang Feng

Zero-shot Cross-domain Dialogue State Tracking via Context-aware Auto-prompting and Instruction-following Contrastive Decoding (2024.emnlp-main)

Challenge:	Existing methods for siMT focus on the Encoder-Decoder architecture, but there are limitations in training and inference.
Approach:	They propose a model that generates translation while reading source tokens . they propose Streaming Self-Attention mechanism tailored for the Decoder-only architecture .
Outcome:	The proposed model achieves state-of-the-art performance on three translation tasks.

Copied to clipboard

Xiaoyu Dong, Yujie Feng, Zexin Lu, Guangyuan Shi, Xiao-Ming Wu

Scalable-DSC: A Structural Template Prompt Approach to Scalable Dialogue State Correction (2023.emnlp-main)

Challenge:	Previous studies have implemented slot-based input improvements, such as schema-driven descriptions and question-answering formats, but still suffer from negative transfer for seen slots and inefficient transfer for unseen slots due to the significant source-target domain gap.
Approach:	They propose a framework that generates dynamic, context-aware slot queries to improve model transferability by penalizing deviations from the provided instructions.
Outcome:	Experiments on two datasets show that the proposed model performs better than existing models on the restaurant domain.

Copied to clipboard

Haoxiang Su, Hongyan Xie, Hao Huang, Shuangyong Song, Ruiyu Fang, Xiaomeng Huang, Sijie Feng

How to Stop an Avalanche? JoDeM: Joint Decision Making through Compare and Contrast for Dialog State Tracking (2022.findings-emnlp)

Challenge:	Existing approaches to correct wrong slot values in dialogue state tracking are intertwined with specific DST models, limiting their applicability to other DSTs.
Approach:	They propose a Scalable Dialogue State Correction model that corrects wrong slot values in predicted dialogue states by using a structural template prompt.
Outcome:	The proposed model achieves state-of-the-art results on MultiWOZ 2.0-2.4.

Copied to clipboard

Haoming Wang, Wang Xin

A Contextual Hierarchical Attention Network with Adaptive Objective for Dialogue State Tracking (2020.acl-main)

Challenge:	Existing state-of-the-art models of dialog state tracking do not address avalanche phenomenon . well-known commercial dialog systems include the Apple Siri, Amazon Alexa, or Microsoft Cortana.
Approach:	They propose a dialog state tracking (DST) model which can tackle the avalanche phenomenon . they propose combining a jointly decision making method and a compare and contrast dialogue update technique .
Outcome:	The proposed model outperforms existing state-of-the-art methods and proves its validity.

Copied to clipboard

Yong Shan, Zekang Li, Jinchao Zhang, Fandong Meng, Yang Feng, Cheng Niu, Jie Zhou

Out-of-Task Training for Dialog State Tracking Models (2020.coling-main)

Challenge:	Existing methods for dialogue state tracking ignore the slot imbalance problem and treat all slots indiscriminately, which limits the learning of hard slots.
Approach:	They propose to employ a contextual hierarchical attention network to enhance the DST by learning contextual representations.
Outcome:	The proposed approach achieves 52.68% and 58.55% joint accuracy on multiWOZ 2.0 and MultiWOZ 2.1 datasets and significantly improves performance (+1.24% and +5.98%)

Copied to clipboard

Michael Heck, Christian Geishauser, Hsien-chin Lin, Nurul Lubis, Marco Moresi, Carel van Niekerk, Milica Gasic

Diable: Efficient Dialogue State Tracking as Operations on Tables (2023.findings-acl)

Challenge:	Dialog state tracking (DST) suffers from data sparsity.
Approach:	They utilize non-dialog data from unrelated NLP tasks to train dialog state trackers . they propose to use dialog state tracking to summarise the conversation history .
Outcome:	The proposed method exploits non-dialog data from unrelated NLP tasks to train dialog state trackers.

Copied to clipboard

Pietro Lesci, Yoshinari Fujinuma, Momchil Hardalov, Chao Shang, Yassine Benajiba, Lluis Marquez

Knowledge-Aware Graph-Enhanced GPT-2 for Dialogue State Tracking (2021.emnlp-main)

Challenge:	Existing systems for dialogue state tracking use the full dialogue history as input and generate the entire state from scratch at each dialogue turn.
Approach:	They propose a task formalisation that represents the dialogue state as a table and formalises it as 'table manipulation task' they represent the dialogue as if it were a list with all the slots and generate the entire state from scratch at each dialogue turn.
Outcome:	The proposed system outperforms existing systems while maintaining competitive accuracy.

Copied to clipboard

Weizhe Lin, Bo-Hsiang Tseng, Bill Byrne

Zero-Shot Dialogue State Tracking via Cross-Task Transfer (2021.emnlp-main)

Challenge:	Existing models for dialogue state tracking are based on Graph Attention Networks . if the relationship between slots and values is modelled explicitly, this can be improved .
Approach:	They propose a model architecture that augments GPT-2 with Graph Attention Networks to allow sequential prediction of slot values.
Outcome:	The proposed architecture improves performance against a strong GPT-2 baseline and with sparsely supervised training.

Copied to clipboard

Zhaojiang Lin, Bing Liu, Andrea Madotto, Seungwhan Moon, Zhenpeng Zhou, Paul Crook, Zhiguang Wang, Zhou Yu, Eunjoon Cho, Rajen Subba, Pascale Fung

Meta-Reinforced Multi-Domain State Generator for Dialogue Systems (2020.acl-main)

Challenge:	Existing approaches to training a dialogue state tracking model require extensive annotated dialogue data.
Approach:	They propose to transfer cross-task knowledge from general question answering corpora to QA model that can handle zero-shot DST.
Outcome:	The proposed model improves existing zero-shot and few-shot results on MultiWoz and shows better generalization ability in unseen domains.

Copied to clipboard

Yi Huang, Junlan Feng, Min Hu, Xiaoting Wu, Xiaoyu Du, Shuo Ma

Conversational Semantic Parsing for Dialog State Tracking (2020.emnlp-main)

Challenge:	Existing methods to train a multi-domain dialogue state tracker are lacking in accuracy.
Approach:	They propose a Meta-Reinforced Multi-Domain State Generator to train a DST meta-learning model with a few domains as source domains and a new domain as target domain.
Outcome:	The proposed system outperforms the traditional training approach with extremely little training data in target domain.

Copied to clipboard

Jianpeng Cheng, Devang Agrawal, Héctor Martínez Alonso, Shruti Bhargava, Joris Driesen, Federico Flego, Dain Kaplan, Dimitri Kartsaklis, Lin Li, Dhivya Piraviperumal, Jason D. Williams, Hong Yu, Diarmuid Ó Séaghdha, Anders Johannsen

PromptAttack: Probing Dialogue State Trackers with Adversarial Prompts (2023.findings-acl)

Challenge:	Language understanding for task-based dialog systems is often termed "dialog state tracking" (DST) whereas semantic parsing is the task of converting a single-turn utterance to a graphstructured meaning representation, DST is more complex.
Approach:	They propose a framework for dialog state tracking that incorporates semantic compositionality, cross-domain knowledge sharing and co-reference.
Outcome:	The proposed framework improves on state-of-the-art approaches for dialog state tracking (DST) it incorporates semantic compositionality, cross-domain knowledge sharing and co-reference.

Copied to clipboard

Xiangjue Dong, Yun He, Ziwei Zhu, James Caverlee

Making Task-Oriented Dialogue Datasets More Natural by Synthetically Generating Indirect User Requests (2025.coling-main)

Challenge:	Toward building more robust and reliable conversational systems, we introduce a prompt-based learning approach to automatically generate effective adversarial examples to probe DST models.
Approach:	They propose a prompt-based learning approach to automatically generate effective adversarial examples to probe DST models.
Outcome:	The proposed framework leads to the greatest reduction in accuracy and the best attack success rate while maintaining good fluency and a low perturbation ratio.

Copied to clipboard

Amogh Mannekote, Jinseok Nam, Ziming Li, Kristy Elizabeth Boyer, Bonnie J. Dorr

Granular Change Accuracy: A More Accurate Performance Metric for Dialogue State Tracking (2024.lrec-main)

Challenge:	Existing task-oriented dialogue benchmarks lack sufficient examples of complex discourse phenomena such as indirectness.
Approach:	They propose a set of linguistic criteria and an LLM-based pipeline for generating realistic IURs to test natural language understanding and dialogue state tracking models before deployment in a new domain.
Outcome:	The proposed model can handle indirect user requests (IURs) but lacks examples of complex discourse phenomena such as indirectness.

Copied to clipboard

Taha Aksu, Nancy Chen

Diverse and Effective Synthetic Data Generation for Adaptable Zero-Shot Dialogue State Tracking (2024.findings-emnlp)

Challenge:	Current metrics for evaluating Dialogue State Tracking (DST) systems exhibit three primary limitations: i) erroneously presume a uniform distribution of slots throughout the dialog; ii) neglect to assign partial scores for individual turns; c) frequently overestimate or underestimate performance by repeatedly counting the models’ successful or failed predictions.
Approach:	They propose a new metric: Granular Change Accuracy (GCA) which evaluates the predicted changes in dialogue state over the entire dialogue history.
Outcome:	The proposed metric reduces biases arising from distribution uniformity and the positioning of errors across turns, resulting in a more precise evaluation.

Copied to clipboard

James Finch, Jinho Choi

Semantic Parsing by Large Language Models for Intricate Updating Strategies of Zero-Shot Dialogue State Tracking (2023.findings-emnlp)

Challenge:	Existing zero-shot dialogue state tracking datasets are limited in the number of domains and slot types they cover due to the high costs of data collection.
Approach:	They propose a fully automatic approach that generates synthetic zero-shot dialogue state tracking datasets.
Outcome:	The proposed approach can generate dialogues across 1,000+ domains with silver-standard dialogue state annotations and slot descriptions.

Copied to clipboard

Yuxiang Wu, Guanting Dong, Weiran Xu

SGP-TOD: Building Task Bots Effortlessly via Schema-Guided LLM Prompting (2023.findings-emnlp)

Challenge:	Existing methods for zero-shot Dialogue State Tracking have focused on domaintransfers and have not yielded satisfactory results.
Approach:	They propose a new In-Context Learning method to introduce additional updating strategies in zero-shot DST by leveraging powerful Large Language Models and translating the original dialogue to JSON through semantic parsing as an intermediate state.
Outcome:	The proposed method outperforms existing zero-shot DST methods on MultiWOZ, showing significant improvements in JGA and slot accuracy compared to existing methods.

Copied to clipboard

Xiaoying Zhang, Baolin Peng, Kun Li, Jingyan Zhou, Helen Meng

S3-DST: Structured Open-Domain Dialogue Segmentation and State Tracking in the Era of LLMs (2024.findings-acl)

Challenge:	Experimental results show that SGP-TOD provides state-of-the-art zero-shot performance . prevailing approach for creating task bots is to fine-tune pre-trained language models .
Approach:	They propose a Schema-Guided Prompting for building Task-Oriented Dialog systems . they use predefined task schema and dialog policy to instruct fixed LLMs to generate appropriate responses .
Outcome:	The proposed system outperforms few-shot approaches on multiwoz, RADDLE, and STAR datasets.

Copied to clipboard

Sarkar Snigdha Sarathi Das, Chirag Shah, Mengting Wan, Jennifer Neville, Longqi Yang, Reid Andersen, Georg Buscher, Tara Safavi

Building Multi-domain Dialog State Trackers from Single-domain Dialogs (2023.emnlp-main)

Challenge:	Dialogue state tracking (DST) was based on narrow task-oriented conversations . however, large language models have ushered in more flexible open-domain chat systems .
Approach:	They propose a method that combines dialogue segmentation and state tracking within open-domain dialogues to improve long context tracking.
Outcome:	The proposed method outperforms the state-of-the-art on open-domain dialogue datasets and publicly available datasets.

Copied to clipboard

Qi Zhu, Zheng Zhang, Xiaoyan Zhu, Minlie Huang

MoPE: Mixture of Prefix Experts for Zero-Shot Dialogue State Tracking (2024.lrec-main)

Challenge:	Existing multi-domain dialog state tracking models require significant manual effort to define domain relations and collect data.
Approach:	They propose a divide-and-conquer (DAC) DST paradigm and a multi-domain dialog synthesis framework to build multi- domain DST models from single-domain dialogues.
Outcome:	The proposed paradigm makes building multi-domain DST models easier on unseen domain combinations.

Copied to clipboard

Tianwen Tang, Tong Zhu, Haodong Liu, Yin Bai, Jia Cheng, Wenliang Chen

Beyond Single-User Dialogue: Assessing Multi-User Dialogue State Tracking Capabilities of Large Language Models (2025.findings-emnlp)

Challenge:	Existing zero-shot dialogue state tracking models suffer from domain transferring and partial prediction problems.
Approach:	They propose to establish connections between similar slots in different domains to improve model transfer performance in unseen domains.
Outcome:	Empirical results show that the proposed model achieves the goal accuracy of 57.13% on MultiWOZ2.1 and 55.4.

Copied to clipboard

Sangmin Song, Juhwan Choi, JungMin Yun, YoungBin Kim

Improving Dialogue State Tracking through Combinatorial Search for In-Context Examples (2025.acl-long)

Challenge:	Large language models have demonstrated remarkable performance in zero-shot dialogue state tracking (DST), reducing the need for task-specific training.
Approach:	They extend existing DST dataset by generating utterances of a second user based on speech act theory.
Outcome:	The proposed model incorporates utterances of a second user into conversations, enabling a controlled evaluation of LLMs in multi-user settings.

Copied to clipboard

Haesung Pyun, Yoonah Park, Yohan Jo

Information Retrieval

Challenge:	Existing methods for training dialogue state tracking data are suboptimal . existing methods rely on suboptimized data, resulting in poor performance .
Approach:	They propose a method that scores effective in-context examples based on their combinatorial impact on DST performance.
Outcome:	The proposed method achieves a 20% gain in data efficiency and generalizing well to the SGD dataset.

Know Your Mistakes: Towards Preventing Overreliance on Task-Oriented Conversational AI Through Accountability Modeling (2025.acl-long)

Copied to clipboard

Suvodip Dey, Yi-Jyun Sun, Gokhan Tur, Dilek Hakkani-Tür