| Challenge: | Neural models of dialog rely on generalized latent representations of language. |
| Approach: | They propose a training procedure which explicitly learns multiple representations of language at several levels of granularity. |
| Outcome: | The proposed training procedure significantly improves performance on the next utterance retrieval task using the MultiWOZ dataset and the Ubuntu dialog corpus. |
Similar Papers
Continuous Decomposition of Granularity for Neural Paraphrase Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Prior work has shown that decomposing sentences at different levels of granularity has improved paragraph generation. |
| Approach: | They propose a model for continuous decomposing granularity for neural paraphrase generation that incorporates granules into attention. |
| Outcome: | The proposed model outperforms baseline models on Quora question pairs and Twitter URLs on two benchmarks. |
Learning to Control the Specificity in Neural Response Generation (P18-1)
Copied to clipboard
| Challenge: | Existing generative conversational models tend to favor general and trivial responses which appear frequently. |
| Approach: | They propose a controlled response generation mechanism to handle different utterance-response relationships in terms of specificity. |
| Outcome: | The proposed model outperforms state-of-the-art models under automatic and human evaluations. |
Pre-training Multi-party Dialogue Models with Latent Discourse Inference (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies have failed to scale up the pre-training process by putting aside unlabeled data . et al., 2019: multi-party dialogues are more difficult for models to understand since they involve multiple interlocutors resulting in interweaving reply-to relations and information flows. |
| Approach: | They propose to treat discourse structures as latent variables and jointly infer them to pre-train a model that understands the discourse structure of multi-party dialogues. |
| Outcome: | The proposed model outperforms baselines and achieves state-of-the-art results on multiple downstream tasks. |
DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation (2022.acl-long)
Copied to clipboard
Wei Chen, Yeyun Gong, Song Wang, Bolun Yao, Weizhen Qi, Zhongyu Wei, Xiaowu Hu, Bartuer Zhou, Yi Mao, Weizhu Chen, Biao Cheng, Nan Duan
| Challenge: | Existing pre-trained dialog models shed light on various downstream tasks in natural language processing (NLP). |
| Approach: | They propose a dialog pre-training framework that introduces latent variables into the enhanced encoder-decoder pre-train framework to increase relevance and diversity of responses. |
| Outcome: | The proposed model achieves state-of-the-art on personaChat, DailyDialog, and DSTC7-AVSD datasets. |
Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation (P18-1)
Copied to clipboard
| Challenge: | Existing encoder-decoder dialog models cannot output interpretable actions as in traditional systems. |
| Approach: | They propose an unsupervised discrete sentence representation learning method that integrates with existing encoder-decoder dialog models for interpretable response generation. |
| Outcome: | The proposed model can be integrated with existing encoder-decoder dialog models and discover interpretable semantics via either auto encoding or context predicting. |
Boosting Dialog Response Generation (P19-1)
Copied to clipboard
| Challenge: | Neural models generate the most common and generic responses all the time . Empirical results show that our method can significantly improve the diversity of responses generated by sequence-to-sequence models. |
| Approach: | They propose an iterative training process and ensemble method based on boosting to improve the diversity of responses generated by neural models. |
| Outcome: | Empirical results show that the proposed method significantly improves diversity and relevance of responses generated by all models. |
Multi-Granularity Contrasting for Cross-Lingual Pre-Training (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to pre-training focus on embedding alignment, but they neglect the modeling of bidirectional contexts. |
| Approach: | They propose a framework to learn languageuniversal representations using multi-granularity contrasting framework . they encode semantic equivalents from different languages into similar representations . |
| Outcome: | The proposed framework can achieve significant performance gains in machine translation and cross-lingual language understanding. |
Pretraining Methods for Dialog Context Representation Learning (P19-1)
Copied to clipboard
| Challenge: | Existing methods for pretraining dialog context encoders are still in their infancy. |
| Approach: | They propose to use unsupervised pretraining objectives for dialog context representations to fine-tune and evaluate them on a set of downstream dialog tasks. |
| Outcome: | The proposed methods improve performance on a set of dialog tasks and are less data hungry. |
Morphology-Aware Multi-Granularity Representation Learning for Agglutinative Languages (2026.acl-srw)
Copied to clipboard
| Challenge: | Existing methods for learning low-resource agglutinative languages are limited to word and phrase levels. |
| Approach: | They propose a morphology-aware gated multi-granularity pre-training framework for agglutinative languages . framework leverages morphological knowledge and integrates a word-level encoder to capture contextual semantics . |
| Outcome: | The proposed framework improves on Mongolian and Turkish agglutinative languages . it leverages morphological knowledge and integrates tagging and segmentation to build fine-grained representations . |
Retrieve & Memorize: Dialog Policy Learning with Multi-Action Memory (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent years have seen a rapid growth of interest in building task-oriented dialogue systems. |
| Approach: | They propose a retrieve-and-memorize framework to deal with unbalanced distribution of system actions in dialogue datasets. |
| Outcome: | The proposed framework achieves competitive performance among state-of-the-art models on a large-scale task-oriented dialogue dataset. |