| Challenge: | Existing versions of MultiWOZ 2.0 have been published, but there are still lots of noisy labels in the training set. |
| Approach: | They propose a framework to train dialogue state tracking models from noisy labels instead of improving annotation quality further by using auxiliary models. |
| Outcome: | The proposed framework improves the goal accuracy of DST models by 28.16% on MultiWOZ 2.0 and 8.41% on MultiWoz 2.4, compared to using only the vanilla noisy labels. |
Similar Papers
MetaASSIST: Robust Dialogue State Tracking with Meta Learning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing dialogue datasets contain lots of noise in their state annotations. |
| Approach: | They propose a framework to train robust dialogue state tracking models by combining pseudo and vanilla labels by a common weighting parameter. |
| Outcome: | The proposed framework achieves state-of-the-art accuracy of 80.10% on multiWOZ 2.4. |
Improving Limited Labeled Dialogue State Tracking with Self-Supervision (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing dialogue state tracking models require plenty of labeled data, but collecting labels is expensive. |
| Approach: | They propose to use only 1% labeled data to train dialogue state tracking models . they encourage a model to have consistent latent distributions given a perturbed input . |
| Outcome: | The proposed self-supervised signals improve goal accuracy by 8.95% when only 1% labeled data is used on the MultiWOZ dataset. |
UNO-DST: Leveraging Unlabelled Data in Zero-Shot Dialogue State Tracking (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for zero-shot dialogue state tracking (DST) ignore unlabelled data in the target domain. |
| Approach: | They propose to transform zero-shot dialogue state tracking into few-shot DST by utilising unlabelled data via joint and self-training methods. |
| Outcome: | The proposed method improves joint goal accuracy by 8% on general language models in zero-shot scenarios, and can be used in many domains. |
Schema Encoding for Transferable Dialogue State Tracking (2022.coling-1)
Copied to clipboard
| Challenge: | Recent work has focused on deep neural models for task-oriented dialogue systems . however, the neural models require a large dataset for training and a new dataset to be trained on another domain. |
| Approach: | They propose a schema encoder for transferable dialogue state tracking to new domains . they aim to transfer the model to new datasets by encoding new schemas based on the dataset . |
| Outcome: | The proposed method improves the accuracy of the proposed model on multi-domain settings. |
Robust Dialogue State Tracking with Weak Supervision and Sparse Data (2022.tacl-1)
Copied to clipboard
Michael Heck, Nurul Lubis, Carel van Niekerk, Shutong Feng, Christian Geishauser, Hsien-Chin Lin, Milica Gašić
| Challenge: | Generalizing dialogue state tracking (DST) to new data and domains is especially challenging due to the strong reliance on abundant and fine-grained supervision during training. |
| Approach: | They propose a training strategy to build extractive DST models without the need for fine-grained manual span labels. |
| Outcome: | The proposed model improves robustness against sample sparsity, new concepts, and topics, leading to state-of-the-art performance on a range of benchmarks. |
Correctable-DST: Mitigating Historical Context Mismatch between Training and Inference for Improved Dialogue State Tracking (2022.emnlp-main)
Copied to clipboard
Hongyan Xie, Haoxiang Su, Shuangyong Song, Hao Huang, Bo Zou, Kun Deng, Jianghua Lin, Zhihui Zhang, Xiaodong He
| Challenge: | Existing dialogue state tracking approaches predict the dialogue state of a target turn sequentially based on the ground-truth previous dialogue state. |
| Approach: | They propose a method that predicts dialogue state sequentially based on previous dialogue state . they propose generating a previously “predicted” dialogue state using ground-truth previous dialogue states . |
| Outcome: | The proposed method achieves 67.51%, 68.24%, 70.30%, 71.38%, and 81.27% joint goal accuracy on MultiWOZ 2.0-2.4 datasets. |
MultiWOZ 2.1: A Consolidated Multi-Domain Dialogue Dataset with State Corrections and State Tracking Baselines (2020.lrec-1)
Copied to clipboard
Mihail Eric, Rahul Goel, Shachi Paul, Abhishek Sethi, Sanchit Agarwal, Shuyang Gao, Adarsh Kumar, Anuj Goyal, Peter Ku, Dilek Hakkani-Tur
| Challenge: | MultiWOZ 2.0 has substantial noise in dialogue state annotations and dialogue utterances . follow-up work has augmented the original dataset with user dialogue acts . |
| Approach: | They propose to reannotate dialogue state and utterances based on original dataset . they then compare their results to other datasets to improve their models . |
| Outcome: | The proposed dataset improves on the noise in the dialogue state annotations and dialogue utterances. |
GCDST: A Graph-based and Copy-augmented Multi-domain Dialogue State Tracking (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to training DST on a single domain ignore information across domains. |
| Approach: | They construct a dialogue state graph to transfer structured features among related domain-slot pairs across domains and encode the graph information of dialogue states by graph convolutional networks. |
| Outcome: | The proposed model improves the performance of the multi-domain DST baseline with the absolute joint accuracy of 2.0% and 1.0% on the MultiWOZ 2.0 and 2.1 dialogue datasets. |
Out-of-Task Training for Dialog State Tracking Models (2020.coling-main)
Copied to clipboard
Michael Heck, Christian Geishauser, Hsien-chin Lin, Nurul Lubis, Marco Moresi, Carel van Niekerk, Milica Gasic
| Challenge: | Dialog state tracking (DST) suffers from data sparsity. |
| Approach: | They utilize non-dialog data from unrelated NLP tasks to train dialog state trackers . they propose to use dialog state tracking to summarise the conversation history . |
| Outcome: | The proposed method exploits non-dialog data from unrelated NLP tasks to train dialog state trackers. |
Enhancing Dialogue State Tracking Models through LLM-backed User-Agents Simulation (2024.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that the model can be used to generate dialogues in new domains quickly. |
| Approach: | They propose to use LLMs to generate dialogue data to reduce dialogue collection and annotation costs. |
| Outcome: | The proposed model performs better than the baseline model trained on real data. |