Robots-Dont-Cry: Understanding Falsely Anthropomorphic Utterances in Dialog Systems (2022.emnlp-main)
Copied to clipboard
| Challenge: | Dialog systems often output human-like responses, but some are impossible for a machine to say. |
| Approach: | They collect ratings on the feasibility of 900 two-turn dialogs from 9 data sources . they build classifiers and explore how modeling configuration might affect output permissibly . |
| Outcome: | The proposed model can be used to train human-like dialogs, but it is not anthropomorphic. |
Similar Papers
The R-U-A-Robot Dataset: Helping Avoid Chatbot Deception by Detecting User Questions About Human or Non-Human Identity (2021.acl-long)
Copied to clipboard
| Challenge: | We analyze 2,500 phrasings related to the intent of “Are you a robot?” and 2,500 adversarially selected utterances to determine whether systems are non-human. |
| Approach: | They analyze 2,500 phrasings related to the intent of "Are you a robot?" and 2,500 adversarially selected utterances to determine whether systems are non-human. |
| Outcome: | The proposed model and two systems fail to confirm non-human intent, and the proposed model is complex. |
From Pixels to Personas: Investigating and Modeling Self-Anthropomorphism in Human-Robot Dialogues (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that robots display human-like characteristics in dialogues . this anthropomorphism raises concerns about the accuracy of AI and its capabilities . |
| Approach: | They propose to use a dataset to analyze self-anthropomorphic and non-self-anthropophilic responses in robots . they propose to combine these two types of responses to create a new category of bot responses . |
| Outcome: | The proposed approach preserves the original dialogues from existing corpora and enhances them with paired responses: self-anthropomorphic and non-self-anthropophilic for each original bot response. |
What is wrong with you?: Leveraging User Sentiment for Automatic Dialog Evaluation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing metrics for dialog evaluation are trained on human annotations, which is cumbersome to collect. |
| Approach: | They propose to use user sentiment and other information as proxy to measure the quality of previous dialogs. |
| Outcome: | The proposed model is comparable to models trained on human annotated data. |
DialogBench: Evaluating LLMs as Human-like Dialogue Systems (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing benchmarks only evaluate LLMs' abilities for task completion as assistant AI. |
| Approach: | They propose a dialogue evaluation benchmark that contains 12 dialogue tasks to evaluate LLMs' capabilities as human-like dialogue systems. |
| Outcome: | The proposed benchmark contains 12 tasks to evaluate LLMs' capabilities . it shows that instruction tuning improves human likeness, but not as human-like systems . |
Mirages. On Anthropomorphism in Dialogue Systems (2023.emnlp-main)
Copied to clipboard
| Challenge: | Automated dialogue systems are anthropomorphised by developers and personified by users. |
| Approach: | They propose to examine linguistic factors that contribute to the anthropomorphism of dialogue systems and the harms that can arise thereof. |
| Outcome: | The proposed systems are anthropomorphised and personified by users . linguistic factors can also be used to reinforce gender stereotypes and conceptions of acceptable language. |
Understanding User Utterances in a Dialog System for Caregiving (2020.lrec-1)
Copied to clipboard
| Challenge: | a dialog system that can monitor the health status of seniors has a huge potential for solving the labor shortage in the caregiving industry in aging societies. |
| Approach: | They are developing a yes/no response classifier and an entailment recognizer to correctly interpret user utterances. |
| Outcome: | The proposed system can correctly interpret user utterances and can monitor the health of seniors. |
Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Study (P19-1)
Copied to clipboard
| Challenge: | Neural generative models are becoming more popular when building conversational agents. |
| Approach: | They propose to study the sensitivity of neural dialog models to unnatural perturbations . they experiment with 10 different types of perturbations on 4 multi-turn dialog datasets . |
| Outcome: | The proposed model is sensitive to unnatural changes or perturbations on 4 multi-turn dialog datasets. |
Dehumanizing Machines: Mitigating Anthropomorphic Behaviors in Text Generation Systems (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies have focused on how text generation systems can lead to harmful outcomes such as over-reliance, emotional dependence, dehumanization, deception, or even physical harm. |
| Approach: | They propose to use an inventory of interventions to help identify possible interventions and provide a conceptual framework to help characterize the landscape of possible interventions. |
| Outcome: | The proposed interventions are based on an inventory of interventions grounded in prior literature and a crowdsourcing study where participants edited system outputs to make them less human-like. |
Real or Robotic? Assessing Whether LLMs Accurately Simulate Qualities of Human Responses in Human-LLM Dialogue (2026.findings-acl)
Copied to clipboard
Jonathan Ivey, Shivani Kumar, Jiayu Liu, Hua Shen, Sushrita Rakshit, Rohan Raju, Haotian Zhang, Aparna Ananthasubramaniam, Junghwan Kim, Bowen Yi, Dustin Wright, Abraham Israeli, Anders Giovanni Møller, Lechen Zhang, David Jurgens
| Challenge: | Recent work has sought to use large language models to simulate human-human and human-LLM interactions. |
| Approach: | They use a large-scale dataset to generate a paired LLM-LLM and human-LLm dialogues from the WildChat dataset and quantify how well they align with their human counterparts. |
| Outcome: | The proposed models perform similarly in simulating English, Chinese, and Russian dialogues. |
Toward Machine Interpreting: Lessons from Human Interpreting Studies (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current speech translation systems are static and do not adapt to real-world situations in ways human interpreters do. |
| Approach: | They propose to model human interpreting using a new language model to improve usability . they argue that there is great potential to adopt many human interpreted principles . |
| Outcome: | The proposed models can be used to improve human interpreting and improve translation performance. |