| Challenge: | a recent study has examined the effects of subjective ideas on group performance in motorsports. |
| Approach: | They collected dialogues between drivers and engineers in motorsports to test this hypothesis . they defined "sensation" as a unique event unfolding in the mind of a speaker . |
| Outcome: | The results show that the more subjective information interlocutors exchange, the better the group performance in collaborative work. |
Similar Papers
Do dialogue representations align with perception? An empirical study (2023.eacl-main)
Copied to clipboard
| Challenge: | masked language models produce stronger correlations than auto-regressive models, but humans and models make different response selection mistakes. |
| Approach: | They propose to use spoken conversation as a model to measure human comprehension behaviour. |
| Outcome: | The proposed model outperforms the model which produces the strongest correlation with human responses. |
Modeling the Quality of Dialogical Explanations (2024.lrec-main)
Copied to clipboard
Milad Alshomary, Felix Lange, Meisam Booshehri, Meghdut Sengupta, Philipp Cimiano, Henning Wachsmuth
| Challenge: | Existing studies have focused on the interaction of explanation moves, dialogue acts, and topics in successful dialogues with expert explainers. |
| Approach: | They construct a corpus of 399 reddit dialogues and analyze interaction flows and explainee quality using two language models that can handle long inputs. |
| Outcome: | The proposed model predicts that the interaction flows between the explainer and the explainee correlate with the quality of the explanations in terms of a successful understanding on the explain's side. |
Collecting and Analyzing Dialogues in a Tagline Co-Writing Task (2024.lrec-main)
Copied to clipboard
| Challenge: | Currently, most studies on dialogue systems focus on problemsolving dialogues and relatively little research has been done on systems that can engage in creative collaboration with users. |
| Approach: | They designed a tagline co-writing task in which two people collaborate to create taglines via text chat and collected dialogue logs, editing logs and questionnaire results. |
| Outcome: | The proposed task involved a tagline co-writing task in which two people collaborate to create taglines via text chat, and collected dialogue logs, editing logs and questionnaire results. |
Knowing What You Know: Calibrating Dialogue Belief State Distributions via Ensembles (2020.findings-emnlp)
Copied to clipboard
Carel van Niekerk, Michael Heck, Christian Geishauser, Hsien-chin Lin, Nurul Lubis, Marco Moresi, Milica Gasic
| Challenge: | Current models for dialogue state tracking only achieve 55% accuracy . however, they lack in performance compared to belief trackers and do not produce well calibrated distributions. |
| Approach: | They propose to calibrate a model for dialogue belief trackers to measure dialogue state accuracy. |
| Outcome: | The proposed model outperforms existing models in terms of accuracy and accuracy. |
Can Visual Dialogue Models Do Scorekeeping? Exploring How Dialogue Representations Incrementally Encode Shared Knowledge (2022.acl-short)
Copied to clipboard
| Challenge: | Existing evaluation methods for visual dialogue models are not consistent . |
| Approach: | They propose a theory-based evaluation method to examine to what degree visual dialogue models incrementally build up representations that do scorekeeping. |
| Outcome: | The proposed method aims to determine to what degree models build up representations that are appropriate to do scorekeeping of shared commitments throughout a visual dialogue. |
EMPATH: An Ensemble Method for Automatic Fine-Grained Turn-Level Dialogue Empathy Evaluation with a Novel Emotional Distance Metric (2026.findings-acl)
Copied to clipboard
| Challenge: | Empathy evaluation metrics are lacking in the competitions, and classical dialogue evaluation metrics require further investigation. |
| Approach: | They propose a framework which combines fine-tuned models, large language models, classical dialogue evaluation metrics, and a novel metric. |
| Outcome: | The proposed framework improves on the WASSA 2024 benchmark and shows a statistically significant 8% improvement on the EX dataset. |
Transferable Dialogue Systems and User Simulators (2021.acl-long)
Copied to clipboard
| Challenge: | a lack of training data is limiting the development of dialogue systems . we develop a framework for creating dialogue data through self-play between agents . |
| Approach: | They propose a framework that can incorporate new dialogue scenarios through self-play between two agents. |
| Outcome: | The proposed framework is highly effective in bootstrapping the performance of two agents in transfer learning. |
The Interplay of Task Success and Dialogue Quality: An in-depth Evaluation in Task-Oriented Visual Dialogues (2021.eacl-main)
Copied to clipboard
| Challenge: | chit-chat and task-oriented dialogue models are evaluated on their task success metric, but the best model is usually chosen based on task success. |
| Approach: | They compare models playing different games to find out which one is best . they find that this discrepancy is model- and task-agnostic . |
| Outcome: | The proposed model can generate utterances that are indistinguishable from human dialogues by learning to ground, encode, and decode words that do not occur frequently in the training set. |
Leveraging Implicit Feedback from Deployment Data in Dialogue (2024.eacl-short)
Copied to clipboard
| Challenge: | Xu et al., 2023) and Bai ed., 2019) use crowdworkers to collect signals from natural dialogue episodes. |
| Approach: | They use the publicly released BlenderBot deployment data to extract signals from conversations to implicitly measure the quality of a machine-generated utterance. |
| Outcome: | The proposed model improves over baseline models, but some proxy signals can lead to undesirable generations. |
FlowDelta: Modeling Flow Information Gain in Reasoning for Conversational Machine Comprehension (D19-58)
Copied to clipboard
| Challenge: | Existing machine comprehension models focus on a single-turn setting and do not account for previous reasoning processes. |
| Approach: | They propose to explicitly model the information gain through the dialogue reasoning . they propose to apply the proposed mechanism to other machine comprehension models . |
| Outcome: | The proposed model achieves state-of-the-art performance in a conversational QA dataset QuAC and a sequential instruction understanding dataset SCONE. |