What makes a good conversation? How controllable attributes affect human judgments (N19-1)
Copied to clipboard
| Challenge: | Existing work on dialogue models for conversational quality is incompletely understanding the relationship between quality and individual attributes. |
| Approach: | They propose to use conditional training and weighted decoding to control four attributes for chit-chat dialogue: repetition, specificity, response-relatedness and question-asking. |
| Outcome: | The proposed methods improve human quality judgments by controlling combinations of these variables. |
Similar Papers
Semantic Space Grounded Weighted Decoding for Multi-Attribute Controllable Dialogue Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Controlling chatbot utterance generation with multiple attributes is a useful but under-studied problem. |
| Approach: | They propose a framework that possesses strong controllability with a weighted decoding paradigm and improves generation quality with an attribute semantics space. |
| Outcome: | The proposed framework achieves high control accuracy with simultaneous control of 3 aspects while producing interesting and sensible responses even in an out-of-distribution robustness test. |
Language Model Transformers as Evaluators for Open-domain Dialogues (2020.coling-main)
Copied to clipboard
| Challenge: | Computer-based systems for communication with humans are a cornerstone of AI research since the 1950s. |
| Approach: | They propose to use transformer neural networks to predict one or more words based on an already given context to provide an efficient, automatic indication of dialogue quality. |
| Outcome: | The proposed language models show that human evaluators have a positive correlation between the output of the models and scores. |
Large Scale Multi-Actor Generative Dialog Modeling (2020.acl-main)
Copied to clipboard
| Challenge: | Non-goal oriented dialog agents typically exhibit inconsistent personality across conversations or the average personality of all users. |
| Approach: | They propose a model that conditionally models past conversations to probabilistically model multi-turn conversations in the actor’s persona. |
| Outcome: | The proposed model improves perplexity on 1.7M held out Reddit conversations by 0.47 on scaling from 117M to 8.3B parameters. |
Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Features (2021.acl-long)
Copied to clipboard
| Challenge: | Existing systems that strive to be informative teachers are difficult to build . knowledge grounded dialogue systems are difficult because of limited training objectives . |
| Approach: | They propose to train a generative neural dialogue model that is controlled to stay faithful to evidence . they propose to use additional inputs to generate more objective responses . |
| Outcome: | The proposed model produces responses that are perceived by humans to be objective and faithful to evidence. |
Leveraging Implicit Feedback from Deployment Data in Dialogue (2024.eacl-short)
Copied to clipboard
| Challenge: | Xu et al., 2023) and Bai ed., 2019) use crowdworkers to collect signals from natural dialogue episodes. |
| Approach: | They use the publicly released BlenderBot deployment data to extract signals from conversations to implicitly measure the quality of a machine-generated utterance. |
| Outcome: | The proposed model improves over baseline models, but some proxy signals can lead to undesirable generations. |
Human-like informative conversations: Better acknowledgements using conditional mutual information (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing chatbots generate responses that are non-specific w.r.t. one of the contexts, typically the conversational history. |
| Approach: | They propose to build a dialogue agent that can weave new factual content into conversations as naturally as humans. |
| Outcome: | The proposed method trades off pmi for pcmi_h and is preferred by humans for overall quality over the Max-PMI baseline 60% of the time. |
You Impress Me: Dialogue Generation via Mutual Persona Perception (2020.acl-main)
Copied to clipboard
| Challenge: | Existing chit-chat systems tend to generate uninformative responses and lack coherent personality traits due to the diversity of speakers. |
| Approach: | They propose a transmitter-receiver framework which explicitly models understanding between interlocutors. |
| Outcome: | The proposed framework improves on a large public dataset, Persona-Chat, with a significant boost over the state-of-the-art frameworks. |
Reducing Conversational Agents’ Overconfidence Through Linguistic Calibration (2022.tacl-1)
Copied to clipboard
| Challenge: | Neural generative open-domain english-language dialogue agents are currently unsuitable for applications other than entertainement and research. |
| Approach: | They propose to incorporate metacognitive features into the training of a controllable generation model to improve likelihood of correctness. |
| Outcome: | The proposed model improves likelihood of correctness by incorporating metacognitive features into the training of a controllable generation model. |
Substance over Style: Evaluating Proactive Conversational Coaching Agents (2025.acl-long)
Copied to clipboard
Vidya Srinivas, Xuhai Xu, Xin Liu, Kumar Ayush, Isaac Galatzer-Levy, Shwetak Patel, Daniel McDuff, Tim Althoff
| Challenge: | Recent NLP research has focused on single-turn tasks with well-defined objectives or evaluation criteria. |
| Approach: | They describe five multi-turn coaching agents that exhibit distinct conversational styles and evaluate them through a user study. |
| Outcome: | The authors compare user feedback with third-person evaluations from health experts and an LM to find that stylistic components in absence of core functionality are viewed negatively. |
Controllable Conversation Generation with Conversation Structures via Diffusion Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Current generation models fail to effectively utilize rich linguistic and world knowledge to generate coherent long text. |
| Approach: | They propose a conversation generation framework that incorporates human knowledge and conversation structures with both controllability and interpretability for better conversation generation. |
| Outcome: | The proposed framework incorporates human knowledge and conversation structures with both controllability and interpretability for better conversation generation. |