SaFeRDialogues: Taking Feedback Gracefully after Conversational Safety Failures (2022.acl-long)
Copied to clipboard
| Challenge: | Existing open-domain conversational models can easily be made to talk in inadequate ways. |
| Approach: | They propose a task and dataset of graceful responses to safety feedback . they collect 8k dialogues demonstrating safety failures, feedback signaling them, and a response acknowledging feedback. |
| Outcome: | The proposed model improves on a dataset of 8k dialogues demonstrating safety failures, feedback signaling them, and a response acknowledging the feedback. |
Similar Papers
SafeConv: Explaining and Correcting Conversational Unsafe Behavior (2023.acl-long)
Copied to clipboard
| Challenge: | Existing datasets do not provide enough annotation to explain unsafe behavior . current chatbots generate toxic and offensive responses, which can be dangerous . |
| Approach: | They construct a dataset called SafeConv that provides comprehensive annotations for chatbots . they compare safe alternatives to rewrite unsafe responses . |
| Outcome: | The proposed model can explain unsafe behavior and detoxify chatbots, the authors show . the proposed model is able to detect unsafe utterances, extract unsafe spans, and convert unsafe responses to safe versions. |
GrounDial: Human-norm Grounded Safe Dialog Response Generation (2024.findings-eacl)
Copied to clipboard
| Challenge: | Recent conversational AI systems generate unsafe responses agreeing to offensive user input or including toxic content. |
| Approach: | They propose a method where response safety is achieved by grounding responses to commonsense social rules without fine-tuning. |
| Outcome: | The proposed approach is quantitatively and qualitatively safer even without additional data or tuning. |
On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark (2022.findings-acl)
Copied to clipboard
Hao Sun, Guangxuan Xu, Jiawen Deng, Jiale Cheng, Chujie Zheng, Hao Zhou, Nanyun Peng, Xiaoyan Zhu, Minlie Huang
| Challenge: | Dialogue safety problems severely limit the real-world deployment of generative conversational models. |
| Approach: | They propose a taxonomy for dialogue safety specifically designed to capture unsafe behaviors in human-bot dialogue settings. |
| Outcome: | The proposed taxonomy captures unsafe behaviors in human-bot dialogue settings with rich context-sensitive unsafe examples. |
Using In-Context Learning to Improve Dialogue Safety (2023.findings-emnlp)
Copied to clipboard
Nicholas Meade, Spandana Gella, Devamanyu Hazarika, Prakhar Gupta, Di Jin, Siva Reddy, Yang Liu, Dilek Hakkani-Tur
| Challenge: | Recent work has highlighted safety issues with large neural-based conversational models. |
| Approach: | They propose a retrieval-based approach for reducing bias and toxicity in chatbot responses . they retrieve demonstrations of safe responses to similar dialogue contexts to generate a response . |
| Outcome: | The proposed method reduces bias and toxicity in three chatbot models . it can be used in compliment to existing dialogue safety approaches, such as RLHF. |
ProsocialDialog: A Prosocial Backbone for Conversational Agents (2022.emnlp-main)
Copied to clipboard
Hyunwoo Kim, Youngjae Yu, Liwei Jiang, Ximing Lu, Daniel Khashabi, Gunhee Kim, Yejin Choi, Maarten Sap
| Challenge: | Existing dialogue systems fail to respond properly to potentially unsafe user utterances . existing systems either ignore or passively agree with unsafe content . |
| Approach: | They introduce a dataset to teach conversational agents to respond to problematic content following social norms. |
| Outcome: | The proposed dataset shows that ProsocialDialog generates more socially acceptable dialogues than existing models. |
SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems (2022.acl-long)
Copied to clipboard
| Challenge: | Several studies discuss the potential harms and benefits of large language models (LLMs) large neural models can replicate and even amplify negative, stereotypical, and derogatory associations in the data. |
| Approach: | They propose to use a first aid kit to assess the safety of conversational AI in various settings . they propose several future directions and discuss ethical considerations . |
| Outcome: | The proposed tools can provide estimates of the relative safety of systems in various settings, but they still have several shortcomings. |
Risk-graded Safety for Handling Medical Queries in Conversational AI (2022.aacl-short)
Copied to clipboard
| Challenge: | Conversational AI systems can engage in unsafe behaviour when handling medical queries that could lead to death. |
| Approach: | They label medical queries with crowdsourced and expert annotations to identify the seriousness of the prompts and recognise the risk types posed by the responses. |
| Outcome: | The results suggest that these tasks can be automated, but caution should be exercised, as errors can potentially be very serious. |
Human-centric dialog training via offline reinforcement learning (2020.emnlp-main)
Copied to clipboard
Natasha Jaques, Judy Hanwen Shen, Asma Ghandeharioun, Craig Ferguson, Agata Lapedriza, Noah Jones, Shixiang Gu, Rosalind Picard
| Challenge: | a novel offline RL method can train dialog models to produce better conversations without the risk of humans teaching it harmful chat behaviors. |
| Approach: | They develop offline reinforcement learning algorithms that use human feedback to train dialog models . they use language similarity, laughter, sentiment, and more to identify positive feedback . |
| Outcome: | The proposed method improves on existing methods with 80 users in an open-domain setting. |
Context-Sensitive Generation of Open-Domain Conversational Responses (C18-1)
Copied to clipboard
| Challenge: | Existing studies on single-turn conversation generation focus on coherence and context-sensitive generation of open-domain conversational responses. |
| Approach: | They propose static and dynamic attention based approaches for context-sensitive generation of open-domain conversational responses. |
| Outcome: | The proposed model outperforms all baselines on automatic and human evaluation on two public datasets. |
Computational Analysis of Conversation Dynamics through Participant Responsivity (2025.emnlp-main)
Copied to clipboard
| Challenge: | Growing literature explores toxicity and polarization in discourse, with comparatively little work on characterizing what makes dialogue prosocial and constructive. |
| Approach: | They develop and evaluate methods for quantifying responsivity through semantic similarity of speaker turns and large language models to identify the relation between two speaker turns. |
| Outcome: | The proposed method is based on semantic similarity of speaker turns and large language models to identify the relation between two speaker turns. |