Papers by Nancy Chen
Aligning Large Language Models with Human Opinions through Persona Selection and Value–Belief–Norm Reasoning (2025.coling-main)
Copied to clipboard
| Challenge: | Current methods for reasoning and predicting human opinions employ role-playing with personae but face two major issues: LLMs are sensitive to even a single irrelevant persona, skewing predictions by up to 30%; and LLM fail to reason strategically over personas. |
| Approach: | They propose a four-step solution modeling which and how to reason with personae, inspired by the Value–Belief–Norm theory. |
| Outcome: | The proposed model improves existing methods by up to 4% by fine-tuning them with COO's data. |
Granular Change Accuracy: A More Accurate Performance Metric for Dialogue State Tracking (2024.lrec-main)
Copied to clipboard
| Challenge: | Current metrics for evaluating Dialogue State Tracking (DST) systems exhibit three primary limitations: i) erroneously presume a uniform distribution of slots throughout the dialog; ii) neglect to assign partial scores for individual turns; c) frequently overestimate or underestimate performance by repeatedly counting the models’ successful or failed predictions. |
| Approach: | They propose a new metric: Granular Change Accuracy (GCA) which evaluates the predicted changes in dialogue state over the entire dialogue history. |
| Outcome: | The proposed metric reduces biases arising from distribution uniformity and the positioning of errors across turns, resulting in a more precise evaluation. |
On Context Utilization in Summarization with Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models excel in abstractive summarization tasks, delivering fluent and pertinent summaries. |
| Approach: | They conduct the first comprehensive study on context utilization and position bias in summarization. |
| Outcome: | The proposed benchmark compares two methods to alleviate position bias in summarization tasks. |
Personality-aware Student Simulation for Conversational Intelligent Tutoring Systems (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing large language models (LLMs) can be adopted as tutoring agents for math and language learning. |
| Approach: | They propose a framework to construct profiles of different student groups by refining and integrating both cognitive and noncognitive aspects, and leverage LLMs for personality-aware student simulation in a language learning scenario. |
| Outcome: | The proposed framework can construct profiles of different student groups by refining and integrating both cognitive and noncognitive aspects, and leverage LLMs for personality-aware student simulation in a language learning scenario. |
In2Core: Leveraging Influence Functions for Coreset Selection in Instruction Finetuning of Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit surprising abilities across a variety of language tasks. |
| Approach: | They propose an algorithm which selects a coreset by analyzing correlation between training and evaluation samples with a trained model. |
| Outcome: | The proposed algorithm can achieve similar performance with just 50% of the training data while preserving the accuracy of the existing model. |
Multimodal Transformer Networks for End-to-End Video-Grounded Dialogue Systems (P19-1)
Copied to clipboard
| Challenge: | Existing work on video-grounded dialogue systems is limited by feature space and semantic information. |
| Approach: | They propose multimodal transformer networks to encode videos and incorporate information from different modalities. |
| Outcome: | The proposed system generates appropriate conversational response to queries of humans based on visual and audio aspects of a given video . it also generalizes to another multimodal visual-grounded dialogue task, and obtains promising performance. |
Multimodal neural pronunciation modeling for spoken languages with logographic origin (D18-1)
Copied to clipboard
| Challenge: | Graphemes of most languages encode pronunciation, though some are more explicit than others . pronunciation modeling in logographic languages requires decomposing logographs into subunits . |
| Approach: | They propose a multimodal approach to predict pronunciation of Cantonese logographic characters using neural networks. |
| Outcome: | The proposed framework improves performance by 18.1% and 25.0% respectively to unimodal and multimodal baselines. |
Multi-Document Summarization with Centroid-Based Pretraining (2023.acl-short)
Copied to clipboard
| Challenge: | In Multi-Document Summarization, the input is a set of documents, and the output is its summary. |
| Approach: | They propose a novel pretraining objective that uses the ROUGE-based centroid of each document cluster as a proxy for its summary. |
| Outcome: | The proposed model is better or comparable to state-of-the-art models. |
Resilience of Large Language Models for Noisy Instructions (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) are powerful tools for interpreting human commands and generating text. |
| Approach: | They examine the resilience of large language models against five common types of disruptions including ASR, OCR, grammatical errors, typographical errors and distractive content. |
| Outcome: | The models show resistance to noise, but their performance suffers . authors evaluated the models against five common types of disruptions based on their results . |
Multimodal Dialogue State Tracking (2022.naacl-main)
Copied to clipboard
| Challenge: | Dialogue state tracking is a key component of dialogue systems. |
| Approach: | They propose to extend the definition of dialogue state tracking to multimodality . they propose a new synthetic benchmark and a novel baseline for this task . |
| Outcome: | The proposed task is based on a synthetic benchmark and a self-supervised video understanding task. |
VGNMN: Video-grounded Neural Module Networks for Video-Grounded Dialogue Systems (2022.naacl-main)
Copied to clipboard
| Challenge: | Neural module networks (NMN) have been used in image-grounded tasks such as Visual Question Answering (VQA) however, very limited work on NMN has been studied in the video-ground dialogue tasks. |
| Approach: | They propose to use video as the grounding feature in video-grounded dialogues to model the information retrieval process in videogrounded language tasks as a pipeline of neural modules. |
| Outcome: | The proposed model can achieve promising performance on video-grounded dialogue and QA benchmarks. |
Conditional Neural Generation using Sub-Aspect Functions for Extractive News Summarization (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in text summarization have overcome position bias in news articles . however, there are long-standing, unresolved challenges in extractive summarizing . |
| Approach: | They propose a neural framework that can flexibly control summary generation by introducing a set of sub-aspect functions. |
| Outcome: | The proposed framework can flexibly control summary generation by introducing sub-aspect functions . extracted summaries with minimal position bias are comparable with standard models . |
Exploring Self-supervised Logic-enhanced Training for Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Traditional attempts to enhance the logical reasoning abilities of language models often rely on supervised fine-tuning, limiting their generalization to new tasks or domains. |
| Approach: | They propose a framework for integrating logical reasoning capabilities into LLMs and activating them via in-context learning. |
| Outcome: | The proposed framework achieves comparable results to existing models on three language understanding benchmarks. |
UniConv: A Unified Conversational Neural Architecture for Multi-domain Task-oriented Dialogues (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to training dialogue agents separately are not optimized for multi-domain task-oriented dialogues. |
| Approach: | They propose a unified neural architecture for end-to-end conversational systems in multi-domain task-oriented dialogues that jointly trains a bi-level state tracker and a joint dialogue act and response generator. |
| Outcome: | The proposed system outperforms existing systems on the MultiWOZ2.1 benchmark in dialogue state tracking, context-to-text, and end-to end settings. |
N-Shot Learning for Augmenting Task-Oriented Dialogue State Tracking (2022.findings-acl)
Copied to clipboard
| Challenge: | augmentation of task-oriented dialogues has followed standard methods for plain-text despite its richly annotated structure. |
| Approach: | They propose an augmentation framework that utilizes belief state annotations to match turns from various dialogues and form new synthetic dialogues in a bottom-up manner. |
| Outcome: | The proposed framework performs better on seen values and more robust to unseen values on n-shot training scenarios. |
Controllable Neural Dialogue Summarization with Personal Named Entity Planning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that our proposed framework generates fluent and factually consistent summaries under various planning controls using both objective metrics and human evaluations. |
| Approach: | They propose a controllable neural generation framework that can guide dialogue summarization with personal named entity planning. |
| Outcome: | The proposed framework generates fluent and factually consistent summaries under various planning controls using objective metrics and human evaluations. |
Multilingual Neural RST Discourse Parsing (2020.coling-main)
Copied to clipboard
| Challenge: | Existing studies on text discourse parsing for English are limited due to the lack of annotated data. |
| Approach: | They propose to use multilingual vector representations and segment-level translation to establish a neural, cross-lingual discourse parser. |
| Outcome: | The proposed model achieves state-of-the-art on cross-lingual, document-level discourse parsing on all sub-tasks. |
Prompt Optimization via Adversarial In-Context Learning (2024.acl-long)
Copied to clipboard
Do Long, Yiran Zhao, Hannah Brown, Yuxi Xie, James Zhao, Nancy Chen, Kenji Kawaguchi, Michael Shieh, Junxian He
| Challenge: | Existing methods to optimize prompts for in-context learning are based on adversarial learning and are computationally efficient and extensible to other LLMs and tasks. |
| Approach: | They propose a method to optimize prompts for in-context learning by a generator and a discriminator. |
| Outcome: | The proposed method improves state-of-the-art prompt optimization techniques on 13 generation and classification tasks including summarization, arithmetic reasoning, machine translation, data-to-text generation, and the MMLU and big-bench hard benchmarks. |
Adapter-TST: A Parameter Efficient Method for Multiple-Attribute Text Style Transfer (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies explore performing text style transfer on attributes like age, gender, formality, politeness, and formality. |
| Approach: | They propose a framework that freezes the pre-trained model’s original parameters and enables the development of a multiple-attribute text style transfer model. |
| Outcome: | The proposed model outperforms state-of-the-art models on sentiment transfer and multiple-attribute transfer tasks with significantly less computational resources. |
LOCOST: State-Space Models for Long Document Abstractive Summarization (2024.eacl-long)
Copied to clipboard
Florian Le Bronnec, Song Duong, Mathieu Ravaut, Alexandre Allauzen, Nancy Chen, Vincent Guigue, Alberto Lumbreras, Laure Soulier, Patrick Gallinari
| Challenge: | State-space models are a low-complexity alternative to transformers for text generation . however, the quadratic complexity of the input length restricts the application of large pretrained models to long texts. |
| Approach: | They propose an encoder-decoder architecture based on state-space models for conditional text generation with long context inputs. |
| Outcome: | The proposed model saves memory and memory during training and inference time while saving 50% and 87% of memory. |
Analyzing Code Embeddings for Coding Clinical Narratives (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent work on automated ICD coding learn mappings between low-dimensional representations of clinical text reports and codes. |
| Approach: | They propose novel neural networks for encoding medical codes based on textual, structural and statistical characteristics using a single deep learning baseline model. |
| Outcome: | The proposed methods improve the accuracy of medical codes based on their textual, structural and statistical characteristics. |
Joint Dialogue Topic Segmentation and Categorization: A Case Study on Clinical Spoken Conversations (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Utilizing natural language processing in clinical conversations is effective to improve the efficiency of workflows for medical staff and patients. |
| Approach: | They propose a model for dialogue segmentation and topic categorization that integrates natural language processing techniques into a joint model. |
| Outcome: | The proposed model improves on follow-up calls for diabetes management and reduces computational complexity and cost. |
Reading Turn by Turn: Hierarchical Attention Architecture for Spoken Dialogue Comprehension (P19-1)
Copied to clipboard
| Challenge: | Existing research on multi-turn spoken conversations focuses on reading comprehension of passages . interactivity of spoken content can cause lower information density and topic diffusion . |
| Approach: | They propose a hierarchical attention neural network architecture to improve spoken dialogue comprehension by combining turn-level and word-level attention mechanisms. |
| Outcome: | The proposed approach outperforms baseline attention models and is robust to lengthy and out-of-distribution test samples. |
DnA-Eval: Enhancing Large Language Model Evaluation through Decomposition and Aggregation (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are scalable and economical evaluators, but how reliable they are is still under-explored. |
| Approach: | They propose a framework which breaks down the evaluation process into decomposition and aggregation stages based on pedagogical practices and provides an interpretable window for how well LLMs evaluate . |
| Outcome: | The proposed framework improves performance on a variety of meta-evaluation benchmarks by providing an interpretable window for how well LLMs evaluate . |
Would you Rather? A New Benchmark for Learning Machine Alignment with Cultural Values and Social Preferences (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies on optimal decision-making are limited and only consider individuals in isolation. |
| Approach: | They propose a task and corpus for learning alignments between machine and human preferences based on a gamified voting game . |
| Outcome: | The proposed task and corpus show that current state-of-the-art NLP models still leave much room for improvement. |
Set to Ordered Text: Generating Discharge Instructions from Medical Billing Codes (D19-1)
Copied to clipboard
| Challenge: | a neural architecture learns to generate content in a specific order without explicit specifications of the relations between input entities and output entities. |
| Approach: | They propose a natural language generation task that generates discharge instructions from ICD codes . they propose to model content ordering and text generation in a specific order . |
| Outcome: | The proposed model outperforms baseline models in BLEU scores and human evaluation. |
Discursive Socratic Questioning: Evaluating the Faithfulness of Language Models’ Understanding of Discourse Relations (2024.acl-long)
Copied to clipboard
| Challenge: | Discursive Socratic Questioning (DISQ) assesses a model's understanding of discourse relations by requiring systematic accuracy over multiple questions. |
| Approach: | They propose a method that evaluates faithfulness of understanding discourse based on question answering. |
| Outcome: | The proposed method evaluates the faithfulness of understanding discourse based on question answering. |
Prompter: Zero-shot Adaptive Prefixes for Dialogue State Tracking Domain Adaptation (2023.acl-long)
Copied to clipboard
| Challenge: | Parameter-Efficient Transfer Learning (PETL) has the potential to address this problem, but it has yet to be applied to the zero-shot domain adaptation. |
| Approach: | They propose to use descriptions of target domain slots to generate dynamic prefixes that are concatenated to the key and values at each layer’s self-attention mechanism. |
| Outcome: | The proposed method outperforms previous methods on the MultiWOZ and SGD benchmarks. |
Unsupervised Summarization Re-ranking (2023.findings-acl)
Copied to clipboard
| Challenge: | Abstractive summarization models have been gaining popularity, but performance of unsupervised models still lags behind supervised models. |
| Approach: | They propose to re-rank summary candidates in an unsupervised manner to close the performance gap between unsupervised and supervised models. |
| Outcome: | The proposed model improves unsupervised models by up to 7.27% and ChatGPT by up 6.86% relative mean ROUGE across four widely-adopted summarization benchmarks. |
SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarization (2022.acl-long)
Copied to clipboard
| Challenge: | Sequence-to-sequence neural networks have enabled great progress in abstractive summarization. |
| Approach: | They propose to train a second-stage model performing re-ranking on a set of summary candidates by using a mixture of experts. |
| Outcome: | The proposed model outperforms the base model on CNN- DailyMail, XSum and Reddit TIFU with a base PEGASUS. |
Uncertainty Modeling for Machine Comprehension Systems using Efficient Bayesian Neural Networks (2020.coling-industry)
Copied to clipboard
| Challenge: | Neural approaches have improved machine comprehension tasks, but models often operate as a black-box, resulting in lower interpretability. |
| Approach: | They propose a hybrid approach to quantify model uncertainty using Bayesian weight approximation and boost up inference speed by 80% relative to test time. |
| Outcome: | The proposed approach boosts inference speed by 80% relative to the previous approach and is applied to a clinical dialogue comprehension task. |
Multi-expert Prompting Improves Reliability, Safety and Usefulness of Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing enhancements of ExpertPrompting improve the large language model generation process. |
| Approach: | They propose a novel enhancement of ExpertPrompting to improve LLM generation by simulating multiple experts, aggregating their responses and selecting the best among individual and aggregated responses. |
| Outcome: | The proposed enhancement outperforms ExpertPrompting and comparable baselines in truthfulness, factuality, informativeness, usefulness and harmfulness. |
Tractable & Coherent Multi-Document Summarization: Discrete Optimization of Multiple Neural Modeling Streams via Integer Linear Programming (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Multi-document summarization generates summary of corpus of documents consisting of related topics. |
| Approach: | They propose a generic framework to jointly consider coherence and informativeness in multi-document summarization and offers provisions to replace individual components based on the domain of source text. |
| Outcome: | The proposed framework consistently performs better than baselines for objective metrics and human evaluation. |
Modeling What-to-ask and How-to-ask for Answer-unaware Conversational Question Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to generate conversational question are naive and do not account for the answer span. |
| Approach: | They propose a framework for generating a conversational question from a context. |
| Outcome: | The proposed framework achieves state-of-the-art in two different settings compared to existing models . it uses a sentence as the rationale and extracts the answer span from it . |
Towards Summary Candidates Fusion (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for abstractive summarization are limited by the quality of the first-stage candidates. |
| Approach: | They propose a method that fuses several summary candidates to produce a novel abstractive second-stage summary. |
| Outcome: | The proposed method improves ROUGE scores and qualitative properties of fused summaries on several summarization datasets. |
Learning from Bootstrapping and Stepwise Reinforcement Reward: A Semi-Supervised Framework for Text Style Transfer (2022.findings-naacl)
Copied to clipboard
| Challenge: | Text style transfer is an important task in controllable language generation due to the scarcity of large-scale parallel data. |
| Approach: | They propose a semi-supervised framework for text style transfer that bootstraps with supervision guided by automatically constructed pseudo-parallel pairs and improves the sequence-to-sequence policy gradient via reinforcement rewards. |
| Outcome: | The proposed framework achieves state-of-the-art performance on multiple datasets and produces effective generation with as minimal as 10% of training data. |
Guiding Computational Stance Detection with Expanded Stance Triangle Framework (2023.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that strategically-enriched data can significantly improve the performance on out-of-domain and cross-target evaluation. |
| Approach: | They propose to decompose a stance detection task from a theoretical perspective and extend it with additional annotations. |
| Outcome: | The proposed task improves performance on out-of-domain and cross-target evaluations using a linguistic framework. |
Learning Planning-based Reasoning by Trajectories Collection and Process Reward Synthesizing (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have raised concerns regarding the hallucination and flaws in their reasoning process. |
| Approach: | They propose a framework to learn planning-based reasoning through Direct Preference Optimization on collected trajectories, which are ranked according to synthesized process rewards. |
| Outcome: | The proposed model surpasses GPT-3.5-Turbo on logical reasoning benchmarks on a set of logically-based reasoning tasks. |
Multi-label and Multi-target Sampling of Machine Annotation for Computational Stance Detection (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for data-driven annotations require domain-specific and task-aligned supervision. |
| Approach: | They propose a multi-label and multi-target sampling strategy to optimize the annotation quality. |
| Outcome: | The proposed method significantly improves performance and learning efficacy on the benchmark stance detection corpora. |
BiST: Bi-directional Spatio-Temporal Reasoning for Video-Grounded Dialogues (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to video-grounded dialogues focus on superficial temporal-level visual cues, but neglect more fine-grained spatial signals from videos. |
| Approach: | They propose a vision-language neural framework for high-resolution queries in videos based on textual cues that exploits both spatial and temporal-level information. |
| Outcome: | The proposed approach outperforms previous approaches on the TGIF-QA benchmark and significantly outperformed previous approaches. |
CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Annotated data plays a critical role in training models and evaluating their performance. |
| Approach: | They propose a paradigm for Human-LLM co-annotation of unstructured texts at scale that utilizes uncertainty to estimate LLMs’ annotation capability. |
| Outcome: | The proposed model outperforms existing models on many text-annotation tasks with up to 21% performance improvement over random baseline. |
Instructive Dialogue Summarization with Query Aggregations (2023.emnlp-main)
Copied to clipboard
| Challenge: | Conventional dialogue summarization methods generate summaries without considering user’s specific interests. |
| Approach: | They propose a three-step approach to synthesize high-quality query-based summarization triples by training a unified model on three summarizing datasets with multi-purpose instructive triples. |
| Outcome: | The proposed model outperforms state-of-the-art models and even models with larger sizes on four datasets including dialogue summarization and dialogue reading comprehension. |
Exploiting Discourse-Level Segmentation for Extractive Summarization (D19-54)
Copied to clipboard
| Challenge: | Existing approaches to extract summarize text are based on sentences as the elementary unit, but semantic segments containing supplementary information or descriptive details are often nonessential in the generated summaries. |
| Approach: | They propose to exploit discourse-level segmentation as a finer-grained means to more precisely pinpoint the core content in a document. |
| Outcome: | The proposed method improves extractive summarization performance on CNN/Daily Mail dataset. |
SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning (2024.naacl-long)
Copied to clipboard
| Challenge: | a new benchmark for multilingual foundation models is being developed . brittleness of foundation models in the dimensions of semantics and multilinguality is a key limitation . |
| Approach: | They propose a benchmark for multilingual foundation models, SeaEval . they examine how well these models comprehend cultural practices, nuances, and values . |
| Outcome: | The proposed model can be used to evaluate multilingual and multicultural scenarios. |
DecoMT: Decomposed Prompting for Machine Translation Between Related Languages using Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows the power of few-shot prompting with large language models for tasks like machine translation, summarization, and question answering. |
| Approach: | They propose a few-shot prompting approach that decomposes the translation process into word chunks. |
| Outcome: | The proposed approach outperforms established few-shot prompting models with 8 chrF++ scores across languages. |