Papers by Weiyan Shi
Social Influence Dialogue Systems: A Survey of Datasets and Models For Social Influence Tasks (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing research focuses on task-oriented or open-domain dialogue systems with influence skills. |
| Approach: | They propose to define and introduce a category of social influence dialogue systems that influence users’ cognitive and emotional responses. |
| Outcome: | The proposed system is task-oriented or goal-oriented, but it is not open-domain. |
Selective Differential Privacy for Language Modeling (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to protect sensitive data from leaking are over-pessimistic and undifferentiated. |
| Approach: | They propose a new privacy notion, selective differential privacy, to provide rigorous privacy guarantees on the sensitive portion of the data to improve model utility. |
| Outcome: | The proposed privacy-preserving mechanism achieves better utility while remaining safe under various privacy attacks compared to baselines. |
Sentiment Adaptive End-to-End Dialog Systems (P18-1)
Copied to clipboard
| Challenge: | Existing methods to train dialog systems only consider semantic inputs and under-utilize other user information. |
| Approach: | They propose to include user sentiment in the end-to-end learning framework to make dialog systems more user-adaptive and effective. |
| Outcome: | The proposed system improves on a bus information search task with sentiment information. |
INSPIRED: Toward Sociable Recommendation Dialog Systems (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on recommendation dialog systems lack a study on communication strategies used by human speakers for making successful and persuasive recommendations. |
| Approach: | They propose to annotate a dataset of human-human movie recommendation dialogs with sociable recommendation strategies. |
| Outcome: | The proposed model outperforms the baseline model in automatic and human evaluation. |
PRAL: A Tailored Pre-Training Model for Task-Oriented Dialog Generation (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to building task-oriented dialog systems require a substantial amount of annotations and thus are labor-intensive. |
| Approach: | They propose a Pre-trainedRole Alternating Language model (PRAL) that is explicitly designed for task-oriented dialog tasks. |
| Outcome: | The proposed model outperforms or is on par with state-of-the-art models on task-oriented dialog tasks. |
NewsInterview: a Dataset and a Playground to Evaluate LLMs’ Grounding Gap via Informational Interviews (2025.acl-long)
Copied to clipboard
Alexander Spangher, Michael Lu, Sriya Kalyan, Hyundong Justin Cho, Tenghao Huang, Weiyan Shi, Jonathan May
| Challenge: | Existing large datasets (1k-10k transcripts) are generated via crowdsourcing and are inherently unnatural. |
| Approach: | They curate a dataset of 40,000 two-person informational interviews from NPR and CNN . they find that LLMs are significantly less likely than human interviewers to use acknowledgements and pivot to higher-level questions. |
| Outcome: | The proposed model is based on 40,000 interviews with journalists and CNN . |
Refine and Imitate: Reducing Repetition and Inconsistency in Persuasion Dialogues via Reinforcement Learning and Human Demonstration (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Persuasion dialogue systems have long-standing problems of dialogue repetition and inconsistency which could impact user experience and impede the persuaded outcome. |
| Approach: | They propose to refine a language model baseline without user simulators and distill sentence-level information about repetition, inconsistency, and task relevance through rewards. |
| Outcome: | The proposed model outperforms state-of-the-art models on automatic metrics and human evaluation results on a donation persuasion task and generates more diverse, consistent and persuasive conversations according to user feedback. |
Social Intelligence Data Infrastructure: Structuring the Present and Navigating the Future (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing work on social intelligence in NLP does not provide a coherent subfield for researchers to analyze and identify research gaps and future directions. |
| Approach: | They build a social AI taxonomy and a data library of 480 NLP datasets to analyze existing datasets and evaluate language models’ performance in different social intelligence aspects. |
| Outcome: | The proposed infrastructure analyzes existing dataset efforts and evaluates language models’ performance in different social intelligence aspects. |
How to Build User Simulators to Train RL-based Dialog Systems (D19-1)
Copied to clipboard
| Challenge: | RL-based dialog systems require interaction with the environment and obtaining real human users to interact with the system is time-consuming and labor-intensive. |
| Approach: | They propose a method to standardize user simulator building to compare dialog system quality using the same set of user simulators. |
| Outcome: | The proposed method can be used by the community to compare dialog system quality using the same set of user simulators fairly. |
Structured Attention for Unsupervised Dialogue Structure Induction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using structured attention, a model can learn dialogue structure in unsupervised fashion. |
| Approach: | They propose to incorporate structured attention layers into a Variational Recurrent Neural Network model with discrete latent states to learn dialogue structure in an unsupervised fashion. |
| Outcome: | The proposed model learns semantic structures similar to templates used to generate a dialogue corpus on two-party datasets and on multi-party dialogues, disentangling dialogues without human annotation. |
Understanding User Resistance Strategies in Persuasive Conversations (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Persuasive dialog systems have various usages, such as donation persuation and physical exercise persulasion. |
| Approach: | They adopt a preliminary framework on persuasion resistance in psychology and build a fine-grained resistance strategy annotation scheme to analyze the persuitee's resistance strategies. |
| Outcome: | The proposed system can understand and address user resistance strategies appropriately. |
When Life Gives You Lemons, Make Cherryade: Converting Feedback from Bad Responses into Good Labels (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing dialogue models are primarily trained on human-human conversations . thumb ups/downs and gold corrections are often sparse in real-life deployment settings . |
| Approach: | They propose a framework to make use of binary and free-form textual human feedback. |
| Outcome: | The proposed framework improves the final dialogue model by using model-corrected replies. |
Seamlessly Integrating Factual Information and Social Content with Persuasive Dialogue (2022.aacl-main)
Copied to clipboard
| Challenge: | Persuasive dialogue systems are designed for chatbots to communicate with and influence users with specific goals. |
| Approach: | They propose a modular dialogue system framework that integrates factual information and social content into persuasive dialogues. |
| Outcome: | The proposed framework is generalizable to any dialogue tasks that have mixed social and task contents. |
AutoReply: Detecting Nonsense in Dialogue with Discriminative Replies (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for dialogue detection make many errors in their own messages . a dataset of long dialogues richly grounded in the game state contains many errors . |
| Approach: | They propose to use an annotated dialogue dataset to generate automatic responses for dialogue models. |
| Outcome: | The proposed model outperforms handcrafted replies and performs on par with supervised learning approaches. |
LEGOEval: An Open-Source Toolkit for Dialogue System Evaluation via Crowdsourcing (2021.acl-demo)
Copied to clipboard
| Challenge: | Currently, researchers use automatic metrics and human evaluation to evaluate dialogue systems. |
| Approach: | They propose to use a Python API to easily evaluate dialogue systems using Amazon Mechanical Turk. |
| Outcome: | The open-source toolkit provides a fast, consistent method for reproducing human evaluation results. |
Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good (P19-1)
Copied to clipboard
| Challenge: | Persuasion agents are a form of communication that can be used to change people's opinions and actions for social good. |
| Approach: | They designed an online persuasion task where one participant was asked to persult the other to donate to a specific charity. |
| Outcome: | The proposed system could change people's opinions and actions for social good. |
Distilling an End-to-End Voice Assistant Without Instruction Training Data (2025.acl-long)
Copied to clipboard
| Challenge: | Recent efforts to train speech-only LLMs have led to models “forging” speech information from text-only models. |
| Approach: | They propose a paradigm for training Speech Large Language Models without instruction data by using the response of a text-only LLM to transcripts as self-supervision. |
| Outcome: | The proposed model generalizes to Spoken Question Answering, Classification, and Translation and achieves a 72% win rate compared with state-of-the-art models like Qwen 2 Audio . |
Decoding Susceptibility: Modeling Misbelief to Misinformation Through a Computational Approach (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on susceptibility to misinformation rely on self-reported beliefs, which can be subject to bias, expensive to collect, and challenging to scale for downstream applications. |
| Approach: | They propose a computational approach to efficiently model users’ latent susceptibility levels by using demographic factors and political ideology as inputs. |
| Outcome: | The proposed model shows that political leanings and other psychological factors exhibit varying degrees of association with susceptibility to COVID-19 misinformation. |
Controllable Mixed-Initiative Dialogue Generation through Prompting (2023.acl-short)
Copied to clipboard
| Challenge: | Mixed initiative dialogue systems allow all interacting agents to initiate actions to control the interaction. |
| Approach: | They propose to prompt large language models as a drop-in replacement for fine-tuning on conditional generation. |
| Outcome: | The proposed prompts improve fine-tuning and ground truth responses . the results show that generated responses are high . |
Unsupervised Dialog Structure Learning (N19-1)
Copied to clipboard
| Challenge: | Current dialog systems require human experts to design the dialog structure, which is time consuming and sometimes insufficient to satisfy various customer needs. |
| Approach: | They propose to extract dialog structure using a modified VRNN model with discrete latent vectors. |
| Outcome: | The proposed model outperforms existing models on the ability to predict unseen data and is faster and more effective in a reinforcement learning setting. |
Zero-shot Persuasive Chatbots with LLM-Generated Strategies and Information Retrieval (2024.findings-emnlp)
Copied to clipboard
Kazuaki Furumai, Roberto Legaspi, Julio Romero, Yudai Yamazaki, Yasutaka Nishimura, Sina Semnani, Kazushi Ikeda, Weiyan Shi, Monica Lam
| Challenge: | Existing methods to improve persuasive chatbots use only a handful of predefined strategies. |
| Approach: | They propose a persuasive chatbot based on large language models that is factual and more persuasive by leveraging many more nuanced strategies. |
| Outcome: | The proposed chatbot is factual and more persuasive by leveraging many more nuanced strategies. |
CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologies (2024.findings-emnlp)
Copied to clipboard
| Challenge: | CultureBank is a knowledge base built upon users’ self-narratives with 12K cultural descriptors sourced from TikTok and 11K from Reddit. |
| Approach: | They construct a pipeline to construct cultural knowledge bases from different online communities on a massive scale. |
| Outcome: | The proposed pipeline improves cultural awareness of language models by evaluating them on two cultural tasks in a zero-shot setting. |
Just Fine-tune Twice: Selective Differential Privacy for Large Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to protect language models from privacy leakage suffer from limited user control and low utility . et al., 2018: a novel framework that achieves SDP for state-of-the-art large transformer-based models. |
| Approach: | They propose a framework that applies differential privacy to large language models . they use redacted in-domain data to fine-tune the model with original in- domain data . |
| Outcome: | The proposed framework achieves strong utility compared to baselines. |