Papers by Nanyun Peng
Detecting Machine-Generated Long-Form Content with Latent-Space Variables (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing zero-shot methods to distinguish machine-generated long-form texts from humans are vulnerable to domain shift including different decoding strategies, variations in prompts, and attacks. |
| Approach: | They propose a method that incorporates abstract elements as key deciding factors by training a latent-space model on sequences of events or topics derived from human-written texts. |
| Outcome: | The proposed method improves on baselines on three domains and significantly improves over existing methods. |
FLAMES: Improving LLM Math Reasoning via a Fine-Grained Analysis of the Data Synthesis Pipeline (2025.findings-emnlp)
Copied to clipboard
Parker Seegmiller, Kartik Mehta, Soumya Saha, Chenyang Tao, Shereen Oraby, Arpit Gupta, Tagyoung Chung, Mohit Bansal, Nanyun Peng
| Challenge: | Recent work improving LLM math reasoning with synthetic data uses unique setups, making comparison of data synthesis strategies impractical. |
| Approach: | They propose a framework for LLM assessment of math reasoning with synthetic data . they use 10 existing data synthesis strategies and multiple other factors to study performance . |
| Outcome: | The proposed data synthesis strategies outperform public datasets on OlympiadBench, CollegeMath, GSMPlus and MATH. |
FOAM: A Follower-aware Speaker Model For Vision-and-Language Navigation (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing speaker-follower models are follower-agnostic and fail to take state of follower into account. |
| Approach: | They propose a speaker-follower model that is constantly updated given follower feedback . they optimize the speaker and obtain its training signals by evaluating the follower on labeled data . |
| Outcome: | The proposed model outperforms strong baseline models on room-to-room and room-across-room datasets. |
Do LLMs Plan Like Human Writers? Comparing Journalist Coverage of Press Releases with LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Journalists engage in multiple steps in news writing that depend on human creativity, such as exploring different “angles” and selecting sources. |
| Approach: | They propose to use large language models to help journalists plan their news coverage . they find that LLMs recommend more creative angles and more informational sources . |
| Outcome: | The proposed models align better with humans when recommending angles, compared with informational sources. |
Metaphor Generation with Conceptual Mappings (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for metaphor generation lack conceptualization of meaning of the metaphors . recent neural models have led to advances in many areas of natural language generation . |
| Approach: | They propose to encode conceptual mappings between cognitive domains to generate metaphoric expressions by embedding verbs into a literal expression and deriving source/target pairs to train a controlled seq-to-seq generation model. |
| Outcome: | The proposed method outperforms existing models in automatic and human evaluations for basic metaphoricity and conceptual metaphor presence. |
Generating similes effortlessly like a Pro: A Style Transfer Approach for Simile Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Literary tropes are at the crux of human imagination and communication. |
| Approach: | They propose to automatically transform similes from reddit to their literal counterparts using common sense knowledge to generate simile models. |
| Outcome: | The proposed method generates 88% novel similes that do not share properties with training data. |
Human-in-the-Loop Synthetic Text Data Inspection with Provenance Tracking (2024.findings-naacl)
Copied to clipboard
| Challenge: | Data augmentation techniques generate low-quality texts with incorrect labels . a new technique is needed to winnow out texts with inaccurate labels based on provenance inspection . |
| Approach: | They develop a data inspection technique that uses provenance inspection and assistive labeling to winnow out texts with incorrect labels. |
| Outcome: | a new human-in-the-loop data inspection technique can winnow out texts with incorrect labels . the technique can reduce human inspection effort by combining provenance inspection and assistive labeling . |
DEGREE: A Data-Efficient Generation-Based Event Extraction Model (2022.naacl-main)
Copied to clipboard
I-Hung Hsu, Kuan-Hao Huang, Elizabeth Boschee, Scott Miller, Prem Natarajan, Kai-Wei Chang, Nanyun Peng
| Challenge: | Existing models for event extraction require expensive human annotations. |
| Approach: | They propose a data-efficient event extraction model that formulates event extraction as a conditional generation problem. |
| Outcome: | The proposed model can be trained with only a few labeled examples. |
Mitigating Bias for Question Answering Models by Tracking Bias Influence (2024.naacl-long)
Copied to clipboard
Mingyu Ma, Jiun-Yu Kao, Arpit Gupta, Yu-Hsiang Lin, Wenbo Zhao, Tagyoung Chung, Wei Wang, Kai-Wei Chang, Nanyun Peng
| Challenge: | Existing literature observes bias in question answering (QA) models, but there is no method to mitigate it. |
| Approach: | They propose an approach to mitigate the bias of question answering models by observing the influence of a query instance on another instance. |
| Outcome: | The proposed method reduces bias level in all 9 bias categories while maintaining comparable QA accuracy. |
Contextual Label Projection for Cross-Lingual Structured Prediction (2024.naacl-long)
Copied to clipboard
| Challenge: | Prior work favors simplified label translation or relying on word-level alignments for label projection. |
| Approach: | They propose a novel approach CLaP which translates text to target language and performs *contextual translation* on the labels using the translated text as the context. |
| Outcome: | The proposed approach improves translation accuracy on two prediction tasks and shows 2.4 F1 improvement for EAE and 1.4 F1 for named entity recognition. |
COM2SENSE: A Commonsense Reasoning Benchmark with Complementary Sentences (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in pretrained language models have shown promising results on commonsense reasoning benchmark datasets. |
| Approach: | They propose a commonsense reasoning benchmark dataset with 4k sentence pairs . they propose 'gamified' model-in-the-loop setup to incentivize challenging samples . |
| Outcome: | The proposed benchmarks show that the proposed model achieves 71% standard accuracy and 51% pairwise accuracy, well below human performance. |
BRIEF: Bridging Retrieval and Inference for Multi-hop Reasoning via Compression (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to augment language models with external knowledge but they are limited by static nature of pre-training data. |
| Approach: | They propose a lightweight approach that compresses retrieved documents into highly dense textual summaries to integrate into in-context RAG. |
| Outcome: | The proposed approach reduces latency and costs while achieving high performance in open-domain questions. |
Improving Zero-Shot Cross-Lingual Transfer Learning via Robust Training (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained multilingual language encoders do not precisely align words and phrases across languages. |
| Approach: | They propose a learning strategy for training robust models by drawing connections between adversarial examples and failure cases of zero-shot cross-lingual transfer. |
| Outcome: | The proposed model can achieve good performance even if representations of different languages are not aligned well. |
MM-PoisonRAG: Disrupting Multimodal RAG with Local and Global Knowledge Poisoning Attacks (2026.acl-long)
Copied to clipboard
Hyeonjeong Ha, Qiusi Zhan, Jeonghwan Kim, Dimitrios Bralios, Saikrishna Sanniboina, Nanyun Peng, Kai-Wei Chang, Daniel Kang, Heng Ji
| Challenge: | Existing research exposes multimodal large language models to knowledge poisoning attacks . localized poisoning attack achieves up to 56% success rate even under restricted access . globalized poison attack completely disrupts model generation to 0% accuracy with just one poisoned content. |
| Approach: | They propose a framework to study the vulnerability of multimodal RAG under knowledge poisoning attacks. |
| Outcome: | The proposed framework exploits two new attack strategies on multimodal RAGs under knowledge poisoning. |
Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answering (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing QA systems do not have commonsense knowledge or cannot reason with it. |
| Approach: | They propose to augment a general commonsense QA framework with a knowledgeable path generator by extrapolating existing paths from a KG with 'state-of-the-art' language model. |
| Outcome: | The generated paths are interpretable, novel, and relevant to the task. |
LLM-A*: Large Language Model Enhanced Incremental Heuristic Search on Path Planning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing path planning algorithms suffer from significant computational and memory inefficiencies as the state space grows . large language models excel in environmental analysis but fall short in detailed spatial and temporal reasoning . |
| Approach: | They propose a new path planning method that synergistically combines A* and LLMs to improve pathfinding efficiency. |
| Outcome: | The proposed method improves pathfinding efficiency while maintaining integrity of path validity in large-scale scenarios. |
Code-Switched Text Synthesis in Unseen Language Pairs (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing efforts on text synthesis for code-switching require training on code-witched texts in the target language pairs. |
| Approach: | They propose a model that synthesizes code-switched texts for language pairs absent from training data by adding an additional code-sharing module to a pre-trained machine translation model. |
| Outcome: | The proposed model synthesizes code-switched texts for language pairs lacking from training data. |
Gender Biases in Automatic Evaluation Metrics for Image Captioning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained evaluation metrics can perpetuate and amplify biases, causing inability to differentiate between biased and unbiased generations. |
| Approach: | They conduct a systematic study of gender biases in image captioning tasks . they show that pretrained models perpetuate and amplify biase . |
| Outcome: | The proposed model-based evaluation metrics have shown good correlations with human judgments in language generation tasks. |
ExPUNations: Augmenting Puns with Keywords and Explanations (2022.emnlp-main)
Copied to clipboard
Jiao Sun, Anjali Narayan-Chen, Shereen Oraby, Alessandra Cervone, Tagyoung Chung, Jing Huang, Yang Liu, Nanyun Peng
| Challenge: | Puns add the challenge of fusing commonsense and world knowledge with the ability to interpret lexical-semantic ambiguity. |
| Approach: | They propose to augment existing datasets with detailed crowdsourced annotations of puns, keywords and fine-grained funniness ratings to challenge current models' ability to understand and generate humor. |
| Outcome: | The proposed tasks include explanation generation to aid with pun classification and keyword-conditioned pun generation to challenge state-of-the-art models' ability to understand and generate humor. |
Beyond Facts- Benchmarking Distributional Reading Comprehension in Large Language Models (2026.findings-acl)
Copied to clipboard
Pei-Fu Guo, Ya An Tsai, Chun-Chia Hsu, Kai-Xin Chen, Yun-Da Tsai, Kai-Wei Chang, Nanyun Peng, Mi-Yen Yeh, Shou-De Lin
| Challenge: | Existing reading comprehension benchmarks focus on factual information, but many real-world tasks require distributional knowledge expressed across text. |
| Approach: | They propose a reading comprehension benchmark for LLMs to evaluate their ability to infer distributional knowledge from natural language. |
| Outcome: | Experiments with multiple LLMs show that the model outperforms baselines, but performance varies widely across distribution types and characteristics. |
LiveCLKTBench: Towards Reliable Evaluation of Cross-Lingual Knowledge Transfer in Multilingual LLMs (2026.acl-long)
Copied to clipboard
Pei-Fu Guo, Yun-Da Tsai, Chun-Chia Hsu, Kai-Xin Chen, Ya An Tsai, Kai-Wei Chang, Nanyun Peng, Mi-Yen Yeh, Shou-De Lin
| Challenge: | Evaluating cross-lingual knowledge transfer in large language models is challenging, as correct answers in a target language may arise either from genuine transfer or from prior exposure during pre-training. |
| Approach: | They propose a pipeline to isolate and measure cross-lingual knowledge transfer by identifying self-contained, time-sensitive knowledge entities from real-world domains and generating factual questions. |
| Outcome: | The proposed pipeline analyzes multiple LLMs across five languages and shows that cross-lingual transfer is strongly influenced by linguistic distance and often asymmetric across language directions. |
Document-level Entity-based Extraction as Template Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Document-level entity-based extraction (EE) tasks extract entity-centric information from unstructured text across multiple sentences. |
| Approach: | They propose a generative framework for two document-level EE tasks: role-filler entity extraction (RE) and relation extraction ( RE). |
| Outcome: | The proposed framework captures cross-entity dependencies and avoids exponential computation complexity of identifying N-ary relations. |
HyperExpan: Taxonomy Expansion with Hyperbolic Representation Learning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing taxonomies have limited coverage due to expensive manual curation process. |
| Approach: | They propose an algorithm that expands existing taxonomies to preserve their structure in a more expressive hyperbolic embedding space and learns to represent concepts and their relations with a hyperbolical Graph Neural Network. |
| Outcome: | The proposed algorithm outperforms baseline models with representation learning in a Euclidean feature space and achieves state-of-the-art performance on the taxonomy expansion benchmarks. |
SPEED++: A Multilingual Event Extraction Framework for Epidemic Prediction and Preparedness (2024.emnlp-main)
Copied to clipboard
Tanmay Parekh, Jeffrey Kwan, Jiarui Yu, Sparsh Johri, Hyosang Ahn, Sreya Muppalla, Kai-Wei Chang, Wei Wang, Nanyun Peng
| Challenge: | Prior studies focused on English posts to provide early warnings for epidemic prediction, but these work focused on non-English posts. |
| Approach: | They propose a multilingual event extraction framework for extracting epidemic event information for any disease and language using 5.1K tweets in four languages. |
| Outcome: | The proposed framework can provide epidemic warnings for COVID-19 in its earliest stages in Dec 2019 (3 weeks before global discussions) and aggregate community epidemic discussions like symptoms and cure measures, aiding misinformation detection and public attention monitoring. |
LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraints (2024.findings-emnlp)
Copied to clipboard
Thomas Palmeira Ferraz, Kartik Mehta, Yu-Hsiang Lin, Haw-Shiuan Chang, Shereen Oraby, Sijia Liu, Vivek Subramanian, Tagyoung Chung, Mohit Bansal, Nanyun Peng
| Challenge: | Recent studies have shown that LLMs struggle with instructions containing multiple constraints. |
| Approach: | They propose a self-correction pipeline that decomposes the original instruction into a list of constraints and uses a Critic model to decide when and where the LLM’s response needs refinement. |
| Outcome: | The proposed model outperforms GPT-4 on RealInstruct and IFEval even with weak feedback. |
VDebugger: Harnessing Execution Feedback for Debugging Visual Programs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Visual programs are executable code generated by large language models to address visual reasoning problems. |
| Approach: | They propose a critic-refiner framework that localizes and debugs visual programs by tracking execution step by step. |
| Outcome: | The proposed framework detects and corrects program errors leveraging detailed execution feedback, improving interpretability and accuracy. |
Measuring Psychological Depth in Language Models (2024.emnlp-main)
Copied to clipboard
Fabrice Harel-Canada, Hanyu Zhou, Sreya Muppalla, Zeynep Yildiz, Miryung Kim, Amit Sahai, Nanyun Peng
| Challenge: | Current evaluations of creative stories focus on objective properties of the text, such as its style, coherence, diversity, and creativity. |
| Approach: | They propose a framework that measures an LLM's ability to produce authentic and narratively complex stories that provoke emotion, empathy, and engagement. |
| Outcome: | The proposed framework shows that humans can consistently evaluate stories based on the PDS (0.72 Krippendorff’s alpha). |
EnDex: Evaluation of Dialogue Engagingness at Scale (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models that measure engagement use expensive human annotas and abstract definitions of the term. |
| Approach: | They propose a human-reaction based model to evaluate dialogue engagingness . they propose combining distant-supervision with a theoretical foundation for engagement . |
| Outcome: | The proposed model is trained on 80k Reddit-based engagement datasets . it uses distant-supervision from human-reaction feedback to evaluate dialogue engagementness . |
ACCENT: An Automatic Event Commonsense Evaluation Metric for Open-Domain Dialogue Systems (2023.acl-long)
Copied to clipboard
| Challenge: | evaluating commonsense in dialogue systems remains an open challenge . despite the success of open-domain dialogue systems, systems struggle to produce commonsensical responses as humans do. |
| Approach: | They propose an event commonsense evaluation metric empowered by commonsensence knowledge bases. |
| Outcome: | The proposed metric achieves higher correlations with human judgments than baselines. |
Fantastic Questions and Where to Find Them: FairytaleQA – An Authentic Dataset for Narrative Comprehension (2022.acl-long)
Copied to clipboard
Ying Xu, Dakuo Wang, Mo Yu, Daniel Ritchie, Bingsheng Yao, Tongshuang Wu, Zheng Zhang, Toby Li, Nora Bradford, Branda Sun, Tran Hoang, Yisi Sang, Yufang Hou, Xiaojuan Ma, Diyi Yang, Nanyun Peng, Zhou Yu, Mark Warschauer
| Challenge: | Existing QA datasets rarely distinguish fine-grained reading skills, such as the understanding of varying narrative elements. |
| Approach: | They propose to use FairytaleQA to generate 10,580 questions based on 278 children-friendly stories to assess model's fine-grained learning skills. |
| Outcome: | The proposed dataset consists of 10,580 questions derived from 278 children-friendly stories, covering seven types of narrative elements or relations. |
Control Large Language Models via Divide and Conquer (2024.emnlp-main)
Copied to clipboard
| Challenge: | Lexically Constrained Generation (LCG) is a crucial task of text generation. |
| Approach: | They propose a Divide and Conquer Generation strategy to enhance LLMs' performance in Lexically Constrained Generation with prompt-based controlling. |
| Outcome: | The proposed strategy shows 90% improvement on the most challenging LCG task. |
Do Models Really Learn to Follow Instructions? An Empirical Study of Instruction Tuning (2023.acl-short)
Copied to clipboard
| Challenge: | Recent studies on instruction tuning (IT) have achieved great performance with zero-shot generalizability to unseen tasks. |
| Approach: | They analyze how models utilize instructions during IT by comparing model training with altered vs. original instructions. |
| Outcome: | The proposed model outperforms naive models in low resource setting. |
Masked Path Modeling for Vision-and-Language Navigation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | A major challenge in vision-and-language navigation is the limited available training data, which hinders the models’ ability to generalize effectively. |
| Approach: | They propose a masked path modeling objective that pretrains an agent using self-collected data for subsequent navigation tasks. |
| Outcome: | The proposed model pretrains an agent using self-collected data for subsequent navigation tasks eliminating the need for external tools. |
SNaRe: Domain-aware Data Generation for Low-Resource Event Detection (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for ED struggle with label noise and domain drift when applied to specialized domains. |
| Approach: | They propose a domain-aware synthetic data generation framework composed of three components: Scout, Narrator, and Refiner. |
| Outcome: | The proposed framework outperforms baseline approaches on three diverse domain ED datasets and achieves average F1 gains of 3-7% in the zero-shot/few-shot settings and 4-20% improvement for multilingual generation. |
DiCoRe: Enhancing Zero-shot Event Detection via Divergent-Convergent LLM Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Understanding the complex event ontology, extracting domain-specific triggers from the passage, and structuring them appropriately overloads and limits the utility of Large Language Models (LLMs). |
| Approach: | They propose a divergent-convergent reasoning framework that decouples the task of ED using Dreamer and Grounder. |
| Outcome: | The proposed framework outperforms baselines on six datasets across five domains and nine LLMs, achieving 4–7% average gains over the best baseline. |
SYNTHIA: Novel Concept Design with Affordance Composition (2025.acl-long)
Copied to clipboard
Hyeonjeong Ha, Xiaomeng Jin, Jeonghwan Kim, Jiateng Liu, Zhenhailong Wang, Khanh Duy Nguyen, Ansel Blume, Nanyun Peng, Kai-Wei Chang, Heng Ji
| Challenge: | Existing studies on concept design using text-to-image models have enabled rapid ideation of novel visual concepts. |
| Approach: | They propose a framework for generating novel, functionally coherent designs based on desired affordances by decomposing concepts into parts and affordance . they also develop a curriculum learning scheme that fine-tunes T2I models to progressively learn affordance composition while maintaining visual novelty. |
| Outcome: | The proposed framework outperforms state-of-the-art models for novelty and functional coherence in human evaluation. |
GENEVA: Benchmarking Generalizability for Event Argument Extraction with Hundreds of Event Types and Argument Roles (2023.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarking datasets for Event Argument Extraction (EAE) cover less than 40 event types and 25 entity-centric argument roles. |
| Approach: | They propose to use a large and diverse EAE ontology to create a semantic role labeling dataset for EAE that incorporates 115 events and 220 argument roles. |
| Outcome: | The proposed ontology concludes with 115 events and 220 argument roles, with a significant portion of roles not being entities. |
ECONET: Effective Continual Pretraining of Language Models for Event Temporal Reasoning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models (PTLMs) have achieved noticeable success on many NLP tasks, but struggle for tasks that require event temporal reasoning. |
| Approach: | They propose a continual pre-training approach that equips PTLMs with targeted knowledge about event temporal relations by focusing on masked-out event and temporal indicators and discriminating sentences from their corrupted counterparts. |
| Outcome: | The proposed framework improves the PTLMs’ fine-tuning performances across five relation extraction and question answering tasks and achieves new or on-par state-of-the-art in most of our downstream tasks. |
Explaining and Improving Contrastive Decoding by Extrapolating the Probabilities of a Huge and Hypothetical LM (2024.emnlp-main)
Copied to clipboard
| Challenge: | Contrastive decoding (CD) improves the next-token distribution of a large expert language model (LM) using a small amateur LM. |
| Approach: | They propose a new unsupervised decoding method called Asymptotic Probability Decoding (APD) that extrapolates the probability curves from the LMs of different sizes to infer the asymptototic probabilities from an infinitely large LM. |
| Outcome: | The proposed method improves the next-token distribution of a large expert language model using a small amateur LM. |
AMPERE: AMR-Aware Prefix for Generation-Based Event Argument Extraction Model (2023.acl-long)
Copied to clipboard
| Challenge: | Existing generation-based EAE models focus on problem re-formulation and prompt design without incorporating additional information that has been shown to be effective for classification-based models. |
| Approach: | They propose to incorporate AMR into generation-based EAE models by generating AMR-aware prefixes for every layer of the generation model. |
| Outcome: | The proposed model generates AMR-aware prefixes for every layer of the generation model and improves the generation. |
AESOP: Paraphrase Generation with Adaptive Syntactic Control (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for paraphrase generation use fixed syntactic structures for all input sentences. |
| Approach: | They propose to add syntactical control to a pretrained language model to generate fluent paraphrases using a retrieval-based selection module. |
| Outcome: | The proposed model achieves state-of-the-art on semantic preservation and syntactic conformation on two benchmark datasets with ground-truth syntaktic control from human-annotated exemplars. |
ACQUIRED: A Dataset for Answering Counterfactual Questions In Real-Life Videos (2023.emnlp-main)
Copied to clipboard
Te-Lin Wu, Zi-Yi Dou, Qingyuan Hu, Yu Hou, Nischal Chandra, Marjorie Freedman, Ralph Weischedel, Nanyun Peng
| Challenge: | despite its importance, there are few datasets that cover multimodal counterfactual reasoning . a dataset focusing on this area is limited because of its limited coverage over synthetic environments . |
| Approach: | They develop a video question answering dataset that provides questions on multimodal reasoning . they ask questions about counterfactual hypotheses over visual events . |
| Outcome: | The proposed dataset shows a significant performance gap between models and humans . it provides questions that span physical, social, and temporal dimensions . |
Active Instruction Tuning: Improving Cross-Task Generalization by Training on Prompt Sensitive Tasks (2023.emnlp-main)
Copied to clipboard
| Challenge: | Instruction tuning (IT) achieves impressive zero-shot generalization results by training large language models on diverse tasks with instructions. |
| Approach: | They propose a framework to identify informative tasks and then actively tune models on selected tasks. |
| Outcome: | The proposed method outperforms baseline strategies for task selection on NIV2 and Self-Instruct datasets. |
Are Large Language Models Capable of Generating Human-Level Narratives? (2024.emnlp-main)
Copied to clipboard
Yufei Tian, Tenghao Huang, Miri Liu, Derek Jiang, Alexander Spangher, Muhao Chen, Jonathan May, Nanyun Peng
| Challenge: | a recent HCI study has pointed to gaps in machine storytelling ability at the global level . authors show that LLMs have less suspense and less tension than human stories . |
| Approach: | They propose a computational framework to analyze narratives through three discourse-level aspects. |
| Outcome: | The proposed framework analyzes narratives through three discourse-level aspects . it shows that LLMs fall short of human abilities in discourse understanding . |
Argument-Aware Approach To Event Linking (2024.findings-acl)
Copied to clipboard
| Challenge: | Prior research in event linking has mainly borrowed methods from entity linking, overlooking distinct features of events. |
| Approach: | They propose an argument-aware method to improve event linking models by augmenting input text with tagged event argument information. |
| Outcome: | The proposed method improves in-KB and out-of-KB queries and training examples. |
SIMMC-VR: A Task-oriented Multimodal Dialog Dataset with Situated and Immersive VR Streams (2023.acl-long)
Copied to clipboard
Te-Lin Wu, Satwik Kottur, Andrea Madotto, Mahmoud Azab, Pedro Rodriguez, Babak Damavandi, Nanyun Peng, Seungwhan Moon
| Challenge: | Existing models lack a large-scale benchmark to capture user–assistant interactions . et al., 2022: 145-160. |
| Approach: | They propose a video-grounded task-oriented dialog dataset that captures real-world AI-assisted user scenarios in VR. |
| Outcome: | The proposed dataset captures real-world AI-assisted user scenarios in VR. |
Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidence (2025.acl-long)
Copied to clipboard
| Challenge: | Notably, when multiple biases combine, models exhibit catastrophic performance degradation, selecting the answer-containing document in less than 10% of cases over a synthetic biased document without the answer. |
| Approach: | They repurpose a relation extraction dataset to quantify the impact of heuristic biases on retrievers like Dragon+ and Contriever. |
| Outcome: | The proposed models exhibit catastrophic performance degradation when multiple biases combine, selecting the answer-containing document in less than 10% of cases over a synthetic biased document without the answer. |
CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learners (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge editing methods fail to generalize updates to multi-hop reasoning tasks . Existing methods only edit single or a few model layers, inadequately integrate updated knowledge into reasoning pathways. |
| Approach: | They propose a circuit-aware method that enhances the effective integration of updated knowledge in large language models by leveraging curated data samples guided by their analysis. |
| Outcome: | The proposed method improves accuracy and accuracy of 20% on the MQuAKE dataset while requiring less memory. |
VALOR-EVAL: Holistic Coverage and Faithfulness Evaluation of Large Vision-Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation methods focus on object hallucinations, focusing on object outputs . current evaluation methods struggle to address subtle semantic distinctions between outputs and reference data . |
| Approach: | They propose a multi-dimensional benchmark covering objects, attributes, and relations . they propose metric that generalizes CHAIR metric and incorporates faithfulness and coverage . |
| Outcome: | The proposed evaluation framework is more comprehensive and better correlated with humans than existing evaluation methods. |
Biomedical Event Extraction with Hierarchical Knowledge Graphs (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Biomedical event extraction is critical in understanding biomolecular interactions described in scientific corpus. |
| Approach: | They propose to integrate domain knowledge from Unified Medical Language System (UMLS) to a pre-trained language model using Graph Edge-conditioned Attention Networks and hierarchical graph representation. |
| Outcome: | The proposed approach achieves 1.41% F1 and 3.19% F1 improvements on the BioNLP 2011 GENIA Event Extraction task. |
REFFLY: Melody-Constrained Lyrics Editing Model (2025.naacl-long)
Copied to clipboard
| Challenge: | Automatic melody-to-lyric (M2L) generation aims to create lyrics that align with a given melody. |
| Approach: | They propose a framework for automatic melody-to-lyric generation that allows for a more flexible approach to creating lyrics from plain text. |
| Outcome: | The proposed framework outperforms baselines Lyra and GPT-4 in musicality and text quality. |
“Kelly is a Warm Person, Joseph is a Role Model”: Gender Biases in LLM-Generated Reference Letters (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are an effective tool to assist individuals in writing documents. |
| Approach: | They examine gender biases in large language models (LLMs)-generated reference letters . they find that models are biased because they are hallucinated . |
| Outcome: | The proposed model-generated reference letters are evaluated on 2 popular LLMs- ChatGPT and Alpaca. |
Do Nuclear Submarines Have Nuclear Captains? A Challenge Dataset for Commonsense Reasoning over Adjectives and Objects (D19-1)
Copied to clipboard
| Challenge: | a dataset of human judgments is used to test the ability to construct models with an understanding of commonsense knowledge. |
| Approach: | They crowdsource sentences that answer a question about adjectives and their transitivity . they build strong baselines for the task using a classification approach . |
| Outcome: | The proposed model outperforms word-level models on commonsense reasoning tasks. |
Evaluating Large Language Models on Controlled Generation Tasks (2023.emnlp-main)
Copied to clipboard
Jiao Sun, Yufei Tian, Wangchunshu Zhou, Nan Xu, Qian Hu, Rahul Gupta, John Wieting, Nanyun Peng, Xuezhe Ma
| Challenge: | Recent studies have looked into the ability of large language models in various benchmark tasks, including question generation, reading comprehension, multilingual and etc. However, few studies investigate the controllability of large languages. |
| Approach: | They propose to compare large language models with state-of-the-start finetuned smaller models to find that large language model controls are comparable to smaller models. |
| Outcome: | The proposed model can meet hard constraints and perform better than state-of-the-art models. |
Scientific Discourse Tagging for Evidence Extraction (2021.eacl-main)
Copied to clipboard
| Challenge: | Primary experimental articles provide the crucial raw material for all subsequent scientific research, but the growing number of scientific literature makes it difficult for domain experts to efficiently utilize them. |
| Approach: | They propose to automatically extract text fragments from primary research papers that describe the evidence presented in that paper's figures and to use them to build models of scientific argument. |
| Outcome: | The proposed method is able to extract text fragments from primary research papers that describe the evidence presented in that paper's figures, and it is transferable to new datasets. |
Joint Event and Temporal Relation Extraction with Shared Representations and Structured Prediction (D19-1)
Copied to clipboard
| Challenge: | Existing systems treat this task as a pipeline of two separate subtasks, i.e., event extraction and temporal relation classification. |
| Approach: | They propose a joint event and temporal relation extraction model with shared representation learning and structured prediction. |
| Outcome: | The proposed method improves both event extraction and temporal relation extraction over state-of-the-art systems. |
Sibylvariant Transformations for Robust Text Classification (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing text transformation techniques are limited in their ability to expand input space . many techniques can artificially expand labeled training sets or test suites, but are class-preserving . |
| Approach: | They propose a concept of sibylvariance to describe transforms that relax the label-preserving constraint and knowably vary the expected class. |
| Outcome: | The proposed transforms can expand input space, but they are limited in their ability to expand . the proposed transform can knowably vary the expected class and lead to more diverse distributions . |
Evaluating Cultural and Social Awareness of LLM Web Agents (2025.findings-naacl)
Copied to clipboard
Haoyi Qiu, Alexander Fabbri, Divyansh Agarwal, Kung-Hsiang Huang, Sarah Tan, Nanyun Peng, Chien-Sheng Wu
| Challenge: | Existing benchmarks often overlook cultural and social awareness . current evaluations focus on task completion, often ignoring the diverse cultural and socio-cultural backgrounds. |
| Approach: | They propose a benchmark to assess LLM agents’ sensitivity to cultural and social norms across two web-based tasks: online shopping and social discussion forums. |
| Outcome: | The proposed framework evaluates LLM agents’ ability to detect and appropriately respond to norm-violating user queries and observations across two web-based tasks. |
Explaining Mixtures of Sources in News Articles (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that language models are essential for long-form article generation. |
| Approach: | They propose a generative process where a source-selection schema is first selected by a journalist, and then sources are chosen based on categories in that schema. |
| Outcome: | The proposed model can predict the most suitable schema given just the headline with reasonable accuracy. |
Identifying Informational Sources in News Articles (2023.emnlp-main)
Copied to clipboard
| Challenge: | Identifying sources of information in news articles is relevant to many tasks in NLP, including misinformation detection and argumentation. |
| Approach: | They propose a task to study compositionality of sources in news articles to understand how they are chosen to complement each other. |
| Outcome: | The proposed dataset can be used to train high-performing models for information detection and source attribution. |
Target Language-Aware Constrained Inference for Cross-lingual Dependency Parsing (D19-1)
Copied to clipboard
| Challenge: | Existing work on cross-lingual dependency parsing focuses on capturing commonalities between source and target languages and overlooking the potential to leverage the linguistic properties of the target languages to facilitate the transfer. |
| Approach: | They propose to use Lagrangian relaxation and posterior regularization techniques to conduct inference with corpus-statistics constraints to capture commonalities between source and target languages. |
| Outcome: | The proposed algorithms improve on 15 and 17 out of 19 target languages. |
Creative Planning with Language Models: Practice, Evaluation and Applications (2025.naacl-tutorial)
Copied to clipboard
| Challenge: | This tutorial explores how planning has been learned and deployed in creative workflows . many human creative tasks involve extensive planning, and actions need to be taken . |
| Approach: | This tutorial explores how planning has been learned and deployed in creative workflows . authors discuss forward and backward learning approaches for planning in LLMs - and evaluation metrics tailored to latent plans . |
| Outcome: | This tutorial examines how planning has been learned and deployed in creative workflows . it discusses forward and backward learning approaches for planning in LLMs - evaluation metrics tailored to latent plans . |
On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark (2022.findings-acl)
Copied to clipboard
Hao Sun, Guangxuan Xu, Jiawen Deng, Jiale Cheng, Chujie Zheng, Hao Zhou, Nanyun Peng, Xiaoyan Zhu, Minlie Huang
| Challenge: | Dialogue safety problems severely limit the real-world deployment of generative conversational models. |
| Approach: | They propose a taxonomy for dialogue safety specifically designed to capture unsafe behaviors in human-bot dialogue settings. |
| Outcome: | The proposed taxonomy captures unsafe behaviors in human-bot dialogue settings with rich context-sensitive unsafe examples. |
EventPlus: A Temporal Event Understanding Pipeline (2021.naacl-demos)
Copied to clipboard
Mingyu Derek Ma, Jiao Sun, Mu Yang, Kung-Hsiang Huang, Nuan Wen, Shikhar Singh, Rujun Han, Nanyun Peng
| Challenge: | Event information is a type of common sense knowledge that helps people understand how stories evolve and provides predictive hints for future events. |
| Approach: | They propose a temporal event understanding pipeline that integrates state-of-the-art components. |
| Outcome: | The proposed pipeline can be easily adapted to other domains, including biomedical domains. |
Harnessing Black-Box Control to Boost Commonsense in LM’s Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent years have seen remarkable progress in massively Pre-Trained Language Models such as GPT-3 . however, their generated outputs lack commonsense at times . |
| Approach: | They propose a framework that steers a frozen Pre-Trained Language Model towards more commonsense generation by training an auxiliary model. |
| Outcome: | The proposed framework produces plausible outputs that incorporate concepts in a meaningful way. |
DiSCoL: Toward Engaging Dialogue Systems through Conversational Line Guided Response Generation (2021.naacl-demos)
Copied to clipboard
| Challenge: | Recent advances in transformer-based language models and their applications to dialogue systems have succeeded to generate fluent and human-like responses. |
| Approach: | They propose to leverage conversational lines as controllable and informative content-planning elements to guide the generation model produce engaging and informative responses. |
| Outcome: | The proposed system generates engaging and informative responses using convlines as controllable and informative content-planning elements. |
BRIEF-Pro: Universal Context Compression with Short-to-Long Synthesis for Fast and Accurate Multi-Hop Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Experiments show that BRIEF-Pro generates more concise and relevant summaries, enhancing performance across small, large, and proprietary language models. |
| Approach: | They propose a universal, lightweight compressor that distills relevant evidence from retrieved documents into a concise summary for seamless integration into in-context RAG. |
| Outcome: | Experiments on four open-domain multi-hop question-answering datasets show that BRIEF-Pro generates more concise and relevant summaries, enhancing performance across small, large, and proprietary language models. |
Vulnerability of LLMs to Vertically Aligned Text Manipulations (2025.acl-long)
Copied to clipboard
| Challenge: | Recent research shows that vertical text input significantly degrades the accuracy of large language models (LLMs) in text classification tasks. |
| Approach: | They investigate the impact of vertical text input on the performance of LLMs . they find that chain of thought reasoning does not help LLM recognize vertical input . |
| Outcome: | The proposed model can significantly mislead models, posing a risk of bypassing detection in real-world scenarios involving harmful or sensitive information. |
On Measures of Biases and Harms in NLP (2022.findings-aacl)
Copied to clipboard
Sunipa Dev, Emily Sheng, Jieyu Zhao, Aubrie Amstutz, Jiao Sun, Yu Hou, Mattie Sanseverino, Jiin Kim, Akihiro Nishi, Nanyun Peng, Kai-Wei Chang
| Challenge: | Recent studies show that natural language processing (NLP) technologies propagate societal biases about demographic groups associated with attributes such as gender, race, and nationality. |
| Approach: | They propose a framework for harms and questions to help practitioners understand biases . they propose measurable measures to detect and mitigate biased groups . |
| Outcome: | The proposed framework provides a framework for harms and questions for practitioners to answer to guide the development of bias measures. |
CaLM: Contrasting Large and Small Language Models to Verify Grounded Generation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to generate grounded responses are prone to errors due to the irrelevancy of input documents. |
| Approach: | They propose a framework that leverages the insight that a robust grounded response should be consistent with information derived solely from its cited sources. |
| Outcome: | Experiments on three open-domain question-answering datasets show that the proposed framework improves performance by 1.5% to 7% without any model fine-tuning. |
NewsEdits: A News Article Revision Dataset and a Novel Document-Level Reasoning Challenge (2022.naacl-main)
Copied to clipboard
| Challenge: | a large dataset of news article revision histories provides clues to narrative and factual evolution in news articles. |
| Approach: | They propose tasks to predict edit-actions performed during version updates . they define article-level edit actions: Addition, Deletion, Edit and Refactor . |
| Outcome: | The proposed dataset is large-scale and multilingual and spans 15 years . it shows that edit-actions are predictable and are likely to be based on factual evolution . |
Creative Natural Language Generation (2023.emnlp-tutorial)
Copied to clipboard
| Challenge: | This tutorial aims to bring awareness of the important and emerging research area of open-domain creative generation. |
| Approach: | They will review recent studies on creative language generation at sentence level as well as longer forms of text. |
| Outcome: | This paper reviews recent studies on creative language generation at sentence level as well as longer forms of text. |
Content Planning for Neural Story Generation with Aristotelian Rescoring (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current approaches to narrative composition are plagued by difficulty in mastering structure, will veer between topics, and lack long-range cohesion. |
| Approach: | They propose a plot-generation language model and a set of rescoring models that implement an aspect of good story-writing as detailed in Aristotle's Poetics. |
| Outcome: | The proposed system improves the quality of the narrative generated from the proposed model and improves its relevance to a given prompt and quality of stories written with our principled plot structure. |
Broaden the Vision: Geo-Diverse Visual Commonsense Reasoning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Generally, commonsense knowledge is correlated with culture and geographic locations and is only shared locally. |
| Approach: | They construct a Geo-Diverse Visual Commonsense Reasoning dataset to test vision-and-language models’ ability to understand cultural and geo-location-specific commonsense. |
| Outcome: | The proposed models perform better in non-Western regions including East Asia, South Asia, and Africa than in the Western regions. |
Zero-shot Sonnet Generation with Discourse-level Planning and Aesthetics Features (2022.naacl-main)
Copied to clipboard
| Challenge: | a sonnet is a fourteen-line poem with rigorous meter-and-rhyme constraints. |
| Approach: | They propose a framework which plans the poem sketch before decoding a sonnet without training on poems . they use a rhyme module, polishing module and a constrained decoding algorithm to impose the meter-and-rhyme constraint . |
| Outcome: | The proposed framework generates sonnets that are coherent and poetic without training on poems . the proposed framework is based on a framework that plans the poem sketch before decoding . |
Mind the Gesture: Evaluating AI Sensitivity to Culturally Offensive Non-Verbal Gestures (2025.acl-long)
Copied to clipboard
| Challenge: | a dataset of 288 gesture-country pairs is used to evaluate AI systems' cultural awareness of offensive gestures and nonverbal signs. |
| Approach: | They use a dataset of 288 gesture-country pairs annotated for offensiveness, cultural significance, and contextual factors across 25 gestures and 85 countries. |
| Outcome: | The proposed dataset analyzes 288 gesture-country pairs across 25 gestures and 85 countries. |
Go Back in Time: Generating Flashbacks in Stories with Event Temporal Prompts (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing systems that generate *flashbacks* are monotonic and lack explicit guidance on how to insert them. |
| Approach: | They propose to use event temporal orders to encode events as temporal prompts . they leverage a Plan-and-Write framework enhanced by reinforcement learning to generate storylines . |
| Outcome: | The proposed method generates more interesting stories with *flashbacks* while maintaining textual diversity, fluency, and temporal coherence. |
QUDSELECT: Selective Decoding for Questions Under Discussion Parsing (2024.emnlp-main)
Copied to clipboard
| Challenge: | Question Under Discussion (QUD) uses implicit questions to reveal discourse relationships between sentences. |
| Approach: | They propose a framework that selectively decodes the QUD dependency structures considering the QUC criteria. |
| Outcome: | The proposed framework outperforms the state-of-the-art baseline models by 9% in human evaluation and 4% in automatic evaluation. |
Sequentially Controlled Text Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Using GPT-2, long documents can ramble and do not follow human-like writing structure. |
| Approach: | They propose a controlled text generation task that generates documents with structure . they use a news article as a dataset to test different degrees of structural awareness . |
| Outcome: | The proposed task generates documents with a structure that is human-like, but long documents lack structure. |
Plot-guided Adversarial Example Construction for Evaluating Open-domain Story Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to generate implausible stories using plots are unnatural and oversimplify the characteristics of implusible machine-generated stories. |
| Approach: | They propose to generate a more comprehensive set of implausible stories using plots . plots are structured representations of controllable factors used to generate stories . |
| Outcome: | The proposed model improves the quality of generated implausible stories using plots . it shows that the evaluation metrics trained on the generated data correlate better with human judgments compared to baselines. |
Re-ReST: Reflection-Reinforced Self-Training for Language Agents (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to fine tune language agents with reasoning-action trajectories require high-quality model-generated samples, which are hard to obtain for challenging language agent tasks. |
| Approach: | They propose a method to employ reflection during inference without ground-truth feedback to improve agents more autonomously. |
| Outcome: | The proposed method improves self-training performance on open-source language agents by 7.6% and 14.1% respectively. |
What Matters for Neural Cross-Lingual Named Entity Recognition: An Empirical Analysis (D19-1)
Copied to clipboard
| Challenge: | Named entity recognition models are challenging for languages with little training data. |
| Approach: | They propose a simple and efficient neural architecture for cross-lingual named entity recognition models. |
| Outcome: | The proposed model achieves competitive performance with the state-of-the-art on two transferable factors: sequential order and multilingual embedding. |
Men Are Elected, Women Are Married: Events Gender Bias on Wikipedia (2021.acl-short)
Copied to clipboard
| Challenge: | Disproportional event distributions can manifest and amplify social stereotypes . researchers have been using NLP tools to analyze corpora for various tasks on online platforms. |
| Approach: | They propose to scrape a corpus of career and personal life descriptions with demographic information from 10,412 celebrities to facilitate the study. |
| Outcome: | The proposed model detects gender biases in a corpus of career and personal life descriptions and calibrates the results using strategically generated templates. |
Improving Pre-trained Vision-and-Language Embeddings for Phrase Grounding (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have focused on the phrase grounding ability of pretrained vision-and-language models, but it is unclear how they can be used for phrase ground. |
| Approach: | They propose to extract phrase-region pairs from pre-trained vision-and-language embeddings and propose four fine-tuning objectives to improve model phrase grounding ability using image-caption data without any supervised grounding signals. |
| Outcome: | The proposed model outperforms baseline models in weakly-supervised and supervised phrase grounding settings on two representative datasets and shows that it is possible to achieve better phrase groundability without sacrificing representation generality. |
Localizing Active Objects from Egocentric Vision with Symbolic World Knowledge (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on active object grounding from an egocentric perspective is focusing on localizing and tracking active objects that undergo major state change as a result of human actions/interactions to the environment without being told exactly what/where to ground. |
| Approach: | They propose to use a narrated egocentric video dataset to localize and track active objects that undergo major state change as a result of human actions/interactions to the environment without being told exactly what/where to ground. |
| Outcome: | The proposed framework leads to 54% improvement in standard metrics on the TREK-150-OPE-Det localization + tracking task, and >7% improvement in all standard metrics. |
Rˆ3: Reverse, Retrieve, and Rank for Sarcasm Generation with Commonsense Knowledge (2020.acl-main)
Copied to clipboard
| Challenge: | Existing work on sarcasm generation focuses on context incongruity, but new work addresses this problem . |
| Approach: | They propose an unsupervised approach for sarcasm generation based on a non-sarcastic input sentence. |
| Outcome: | The proposed method generates sarcasm better than humans 34% of the time and better than a reinforced hybrid baseline 90% of the times. |
ESTER: A Machine Reading Comprehension Dataset for Reasoning about Event Semantic Relations (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent event-centric reading comprehension datasets focus mostly on event arguments or temporal relations. |
| Approach: | They propose a machine reading comprehension dataset that leverages natural language queries to reason about the five most common event semantic relations. |
| Outcome: | The proposed dataset shows that current SOTA systems achieve 22.1%, 63.3% and 83.5% for token-based exact-match, **F1** and event-based **HIT@1** scores. |
How to Make Large Language Models Generate 100% Valid Molecules? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) can learn to perform a wide range of tasks, but generating valid molecules using representations like SMILES is challenging in few-shot settings. |
| Approach: | They propose a language framework that converts invalid SMILES to SELFIES and LLMs as post-hoc correctors to ensure that the molecules generated by LLM are 100% valid. |
| Outcome: | The proposed model performs worse with SELFIES than with SMILES and improves on other metrics. |
SkillVerse : Assessing and Enhancing LLMs with Tree Evaluation (2025.acl-long)
Copied to clipboard
| Challenge: | Language models evolve to tackle complex, multifaceted tasks, requiring granular evaluations . recent studies have focused on leaderboard and benchmark results, but limited interpretability makes it difficult to compare strengths and weaknesses of models. |
| Approach: | They propose an unsupervised tree-structured diagnosis framework for understanding model proficiency in specific abilities with an LLM as a judge. |
| Outcome: | The proposed framework improves model in-context learning and predicts model weaknesses with a 55% success rate compared to the framework without SkillVerse. |
TextEE: Benchmark, Reevaluation, Reflections, and Future Challenges in Event Extraction (2024.findings-acl)
Copied to clipboard
Kuan-Hao Huang, I-Hung Hsu, Tanmay Parekh, Zhiyu Xie, Zixuan Zhang, Prem Natarajan, Kai-Wei Chang, Nanyun Peng, Heng Ji
| Challenge: | Recent studies suggest that event extraction evaluations may not accurately reflect the true performance. |
| Approach: | They propose a standardized, fair, and reproducible benchmark for event extraction . they use standardized scripts and splits for 16 datasets spanning eight domains . |
| Outcome: | The proposed benchmarks show that they struggle to achieve satisfactory performance. |
Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction (2022.acl-long)
Copied to clipboard
| Challenge: | Existing models for zero-shot cross-lingual event argument extraction are based on pre-trained generative language models. |
| Approach: | They propose to use pre-trained generative language models to generate sentences that fill in a template with arguments extracted from the input passage. |
| Outcome: | The proposed model outperforms the current state-of-the-art models on zero-shot cross-lingual EAE. |
Understanding Multimodal Procedural Knowledge by Sequencing Multimodal Instructional Manuals (2022.acl-long)
Copied to clipboard
| Challenge: | Current machine learning methods are incapable of efficiently utilizing multimodal information. |
| Approach: | They propose to use text-and-image alignment to improve machine learning's performance on multimodal event sequencing. |
| Outcome: | The proposed models perform significantly worse than humans on multimodal event sequencing than humans. |
“Nice Try, Kiddo”: Investigating Ad Hominems in Dialogue Responses (2021.naacl-main)
Copied to clipboard
| Challenge: | Ad hominem attacks target a person's character instead of the position the person is maintaining. |
| Approach: | They propose to use salient n-gram similarity as a soft constraint to reduce the amount of ad hominems generated in Twitter conversations. |
| Outcome: | The proposed method reduces the amount of ad hominems generated in human and dialogue system responses to English Twitter posts by using salient n-gram similarity as a soft constraint. |
STORIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets lack rich enough contexts to guide models and evaluations are unreliable for long-form creative text. |
| Approach: | They propose a dataset and evaluation platform built from STORIUM . their dataset contains 6K lengthy stories with fine-grained natural language annotations . |
| Outcome: | The proposed model can be used to generate 6K long stories with fine-grained natural language annotations and a user-generated dataset. |
MERMAID: Metaphor Generation with Symbolism and Discriminative Decoding (2021.naacl-main)
Copied to clipboard
| Challenge: | a new method for generating metaphors is proposed to generate literal sentences . human evaluations show that our best model generates metaphors better than three well-crafted baselines 66% of the time on average. |
| Approach: | They propose a method to automatically construct a parallel corpus by transforming literal sentences to metaphorical ones using commonsense inference and masked language modeling. |
| Outcome: | The proposed method generates metaphors better than baselines 66% of the time on average. |
Sandcastles in the Storm: Revisiting the (Im)possibility of Strong Watermarking (2025.acl-long)
Copied to clipboard
Fabrice Y Harel-Canada, Boran Erol, Connor Choi, Jason Liu, Gary Jiarui Song, Nanyun Peng, Amit Sahai
| Challenge: | Recent theoretical work argues that any watermark can be erased via random walk attacks that perturb text while preserving quality. |
| Approach: | They propose that any watermark can be erased via random walk attacks that perturb text while preserving quality. |
| Outcome: | The proposed method underperforms the theoretical models in large-scale experiments and human-validated assessments. |
Learning Action Conditions from Instructional Manuals for Instruction Understanding (2023.acl-long)
Copied to clipboard
| Challenge: | a weakly supervised task is proposed to extract mentions of preconditions and postconditions of actions from instructional manuals. |
| Approach: | They propose a task dubbed action condition inference which extracts mentions of preconditions and postconditions of actions from instructional manuals. |
| Outcome: | The proposed approach improves on the existing models, but still far behind human performance. |
Con-ReCall: Detecting Pre-training Data in LLMs via Contrastive Decoding (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods analyze training data with member and non-member contexts, overlooking potential insights from both member and not-member. |
| Approach: | They propose a method that leverages asymmetric distributional shifts induced by member and non-member contexts through contrastive decoding to enhance membership inference. |
| Outcome: | The proposed approach outperforms the current state-of-the-art on the WikiMIA benchmark and is robust against various text manipulation techniques. |
Medical Vision-Language Pre-Training for Brain Abnormalities (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing vision-language models lack expertise for medical applications due to the scarcity and complexity of data. |
| Approach: | They propose a pipeline to collect medical image-text aligned data for pretraining from public resources such as PubMed and build a high-performance vision-language model tailored to specific medical tasks. |
| Outcome: | The proposed model is based on a large brain image-text dataset and will be released to the public. |
AmbiPun: Generating Humorous Puns with Ambiguous Context (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for generating homographic puns are heavy-weighted due to the lack of training data. |
| Approach: | They propose a way to generate pun sentences that does not require training on existing puns. |
| Outcome: | The proposed method outperforms baseline models and state-of-the-art models by a large margin. |
Pun Generation with Surprise (N19-1)
Copied to clipboard
| Challenge: | In this paper, we explore creative generation with a focus on puns. |
| Approach: | They propose an unsupervised approach to generating puns using lots of raw text and a surprisal principle. |
| Outcome: | The proposed approach generates puns 30% of the time, doubles the neural generation baseline. |
TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current machine reading comprehension benchmarks have no questions that test temporal phenomena . a new study studies reading comprehension for temporal relations . |
| Approach: | They propose a reading comprehension benchmark built on news snippets and 21k human-generated questions querying temporal relationships. |
| Outcome: | The new reading comprehension benchmark TORQUE achieves an exact-match score of 51% on the test set . the benchmark is built on 3.2k news snippets with 21k human-generated questions . |
Plan, Write, and Revise: an Interactive System for Open-Domain Story Generation (N19-4)
Copied to clipboard
| Challenge: | a neural narrative generation system interacts with humans to generate stories . a recent resurgence of interest in collaborative storytelling has led to new approaches . |
| Approach: | They propose a neural narrative generation system that interacts with humans to generate stories. |
| Outcome: | The proposed system improves story quality and user engagement under time constraints. |
Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescue (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods that edit large language models with updated knowledge can cause side effects on the general abilities of LLMs such as reasoning, natural language inference, and question answering. |
| Approach: | They propose to regularize the edit update weights by imposing constraints on their complexity based on the RElative Change in weighT. |
| Outcome: | The proposed method can significantly mitigate the side effects while maintaining over 94% editing performance. |
MacGyver: Are Large Language Models Creative Problem Solvers? (2024.naacl-long)
Copied to clipboard
Yufei Tian, Abhilasha Ravichander, Lianhui Qin, Ronan Le Bras, Raja Marjieh, Nanyun Peng, Yejin Choi, Thomas Griffiths, Faeze Brahman
| Challenge: | a new study examines the creative problem-solving capabilities of modern LLMs . it provides insight into the constrained problem- solving capabilities of both humans and AI . |
| Approach: | They use an automatically generated dataset to compare and contrast LLMs and humans to find out their creative problem-solving abilities. |
| Outcome: | The proposed dataset compares LLMs and humans in a constrained setting . it shows that humans excel in tasks they are familiar with but struggle with domain-specific knowledge . |
Paraphrase Generation as Unsupervised Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for paraphrase generation rely on labeled datasets or are limited in narrow domains. |
| Approach: | They propose a paradigm for paraphrase generation by treating the task as unsupervised machine translation based on pairs of unlabeled monolingual sentences. |
| Outcome: | The proposed paradigm can generate paraphrases on a large unlabeled monolingual corpus without relying on bilingual sentence pairs. |
On Difficulties of Cross-Lingual Transfer with Order Differences: A Case Study on Dependency Parsing (N19-1)
Copied to clipboard
| Challenge: | Existing studies on crosslingual transfer have focused on word-level information sharing, but words are not independent in sentences; their combinations form larger linguistic units, known as context. |
| Approach: | They propose to use orderagnostic models to transfer word order to distant languages . they train dependency parsers on an English corpus and evaluate their transfer performance on 30 other languages. |
| Outcome: | The proposed model performs better on languages with different word orders than on other languages. |
Guiding Through Complexity: What Makes Good Supervision for Hard Reasoning Tasks? (2025.naacl-long)
Copied to clipboard
| Challenge: | Using weak teacher models to effectively supervise LLMs can improve performance on hard reasoning tasks. |
| Approach: | They propose two data-driven supervision strategies that offer supervision data at different quality levels upon tasks of varying complexity. |
| Outcome: | The proposed methods outperform "perfectly correct" supervision on harder subtasks even when the outcome error rate is high. |
A Unified Framework for Pun Generation with Humor Principles (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for generating homophonic and homographic puns lack the linguistic attributes of successful puns to resolve the split-up in existing work. |
| Approach: | They propose a framework to generate both homophonic and homographic puns to resolve the split-up in existing works by incorporating three linguistic attributes of puns into the language models: ambiguity, distinctiveness, and surprise. |
| Outcome: | The proposed model over strong baselines shows that it can generate both homophonic and homographic puns. |
Rethinking Creativity Evaluation: A Critical Analysis of Existing Creativity Evaluations (2026.eacl-long)
Copied to clipboard
| Challenge: | Creativity measures that distinguish creativity in one domain fail in others, and different metrics disagree on the same data points. |
| Approach: | They examine, analyze, and compare four representative creativity measures across the diverse creative domains, including creative writing, unconventional problem-solving, and research ideation. |
| Outcome: | The measures of creativity across creative domains are compared using a set of human-aligned examples and lack consistency across domains and metrics. |
Towards Controllable Biases in Language Generation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | a new method to induce societal biases in natural language generation is being developed . a method to equalize the amount of biased text across demographics is effective . |
| Approach: | They propose a method to induce societal biases in natural language generation by using demographic inequalities. |
| Outcome: | The proposed method is effective at equalizing biases across demographics while generating less negatively biased text overall. |
AMRFact: Enhancing Summarization Factuality Evaluation with AMR-Driven Negative Samples Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for evaluating factual consistency of abstractive summarization lack coherence or error-type coverage. |
| Approach: | They propose a framework that generates perturbed summaries using Abstract Meaning Representations (AMRs) they use a selection module NegFilter to ensure the quality of the generated negative examples . |
| Outcome: | The proposed framework outperforms existing systems on the AggreFact-SOTA benchmark and provides high error-type coverage. |
Decoupling Task-Solving and Output Formatting in LLM Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Recent studies suggest intertwining task and format instructions with strict formatting requirements can negatively impact LLMs' reasoning capabilities. |
| Approach: | They propose a decoding framework that explicitly decouples format adherence from problem solving. |
| Outcome: | Experiments show that Deco-G consistently gains over prompting and structured generation baselines, with guaranteed format compliance. |
Uncertainty Calibration for Tool-Using Language Agents (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Language agents are increasingly used to perform tasks and interact with a variety of external tools to achieve specific, goal-oriented objectives. |
| Approach: | They propose a tool calibration tool called ProbeCal which recalibrates the internal probabilities of tool-using language agents to better reflect the actual effectiveness of tool. |
| Outcome: | The proposed model significantly improves off-the-shelf language models in tool-using applications. |
Comparing Bad Apples to Good Oranges Aligning Large Language Models via Joint Preference Optimization (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that acquiring human preferences by comparing generations is not effective for large language models. |
| Approach: | They propose a preference optimization objective that elicits preferences jointly over the instruction-response pairs. |
| Outcome: | The proposed approach outperforms prior preference optimizations by 5.2% and 3.3% in summarization and open-ended dialogue datasets. |
METAL: A Multi-Agent Framework for Chart Generation with Test-Time Scaling (2025.acl-long)
Copied to clipboard
| Challenge: | Chart generation requires strong visual design skills and precise coding capabilities that embed the desired visual properties into code. |
| Approach: | They propose a vision-language model-based multi-agent framework for effective automatic chart generation. |
| Outcome: | The proposed framework achieves a 5.2% improvement in the F1 score over the current best chart generation task. |
TAGPRIME: A Unified Framework for Relational Structure Extraction (2023.acl-long)
Copied to clipboard
| Challenge: | Existing models for natural language processing (NLP) do not address common tasks. |
| Approach: | They propose to take a unified view of all the tasks and introduce a model that appends priming words about the condition to the input text. |
| Outcome: | The proposed model is based on ten datasets across five different languages and covers ten tasks that cover ten languages. |
DEAM: Dialogue Coherence Evaluation using AMR-based Semantic Manipulations (2022.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation metrics for incoherent dialogues are insufficient to accurately reflect incohérence . despite the effectiveness of large pretrained language models, not everyone is into this type of work. |
| Approach: | They propose a Dialogue coherence Evaluation metric that uses Abstract Meaning Representation to apply semantic-level Manipulations for incoherent (negative) data generation. |
| Outcome: | The proposed evaluation metric achieves higher correlations with human judgments compared to baseline methods on dialog datasets by significant margins. |
Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models enable them to follow freeform instructions, including imitating generic or specific demographic personas in conversations. |
| Approach: | They propose to investigate persona biases by experimenting with UNIVERSALPERSONA, a model that incorporates both generic and specific personas. |
| Outcome: | The proposed model systematically measures persona biases in harmful expression and harmful agreement. |
Tracking the Newsworthiness of Public Documents (2024.acl-long)
Copied to clipboard
| Challenge: | a new method to model news coverage of local government is needed . we show that newsworthiness predictions can be useful for journalists seeking to keep abreast of local governments. |
| Approach: | They propose a method that explicitly models when and why stories get press attention . they use an annotated corpus of news articles to build models that predict if a policy item will get covered . |
| Outcome: | The proposed model outperforms retrieval-based methods with limited annotated data and language use between corpora. |
DRS: Deep Question Reformulation With Structured Output (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing models like GPT-3 and Instruct-GPT lack the ability to reformulate unanswerable questions. |
| Approach: | They propose a zero-shot method that combines the strengths of LLMs with a DFS-based algorithm to iteratively explore potential entity combinations and constrain outputs using predefined entities. |
| Outcome: | The proposed method outperforms all baselines, including the GPT-3.5 model, on the unanswerable question reformulation task. |
Re3: Generating Longer Stories With Recursive Reprompting and Revision (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has generated short stories of several pages in length, but they are much shorter than typical short stories meant for human consumption. |
| Approach: | They propose a framework to generate long-range plot coherence and relevance by prompting a general-purpose language model and a language model. |
| Outcome: | The proposed framework generates stories of 2000-2500 words, compared to similar-length stories generated directly from the same model. |
Towards Robust NLG Bias Evaluation with Syntactically-diverse Prompts (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Past studies have shown biases in natural language generation systems but there has been little work on evaluating the bias evaluation approaches. |
| Approach: | They propose a method for evaluating biases in natural language generation systems by paraphrasing syntactic prompts with different syntaktic structures and paraphrazing them to evaluate demographic bias. |
| Outcome: | The proposed method is more robust and shows that some syntactic structures prompt more toxic content while others could prompt less biased generation. |
Domain Knowledge Empowered Structured Neural Net for End-to-End Event Temporal Relation Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to extract event temporal relations from text data are limited by hard constraints and large datasets. |
| Approach: | They propose a framework that enhances deep neural network with distributional constraints constructed by probabilistic domain knowledge to improve the baseline neural network models. |
| Outcome: | The proposed framework improves baseline models with strong statistical significance on two widely used datasets in news and clinical domains. |
Unsupervised Melody-to-Lyrics Generation (2023.acl-long)
Copied to clipboard
Yufei Tian, Anjali Narayan-Chen, Shereen Oraby, Alessandra Cervone, Gunnar Sigurdsson, Chenyang Tao, Wenbo Zhao, Tagyoung Chung, Jing Huang, Nanyun Peng
| Challenge: | Existing methods for automatic melody-to-lyric generation are limited due to the limited amount of melody-lyrical aligned data. |
| Approach: | They propose a method for automatic melody-to-lyric generation without training on any aligned melody-lyr data. |
| Outcome: | The proposed model generates high-quality lyrics that are singable, intelligible, and coherent than baseline models. |
Event Detection from Social Media for Epidemic Prediction (2024.naacl-long)
Copied to clipboard
Tanmay Parekh, Anh Mac, Jiarui Yu, Yuxuan Dong, Syed Shahriar, Bonnie Liu, Eric Yang, Kuan-Hao Huang, Wei Wang, Nanyun Peng, Kai-Wei Chang
| Challenge: | Social media is an easy-to-access platform providing timely updates about societal trends and events. |
| Approach: | They propose a framework to extract epidemic-related events from social media posts to provide early warnings. |
| Outcome: | The proposed framework can detect epidemic events for three unseen epidemics of Monkeypox, Zika, and Dengue while existing models fail miserably. |
HypoGen: Hyperbole Generation with Commonsense and Counterfactual Knowledge (2021.findings-emnlp)
Copied to clipboard
| Challenge: | despite its abundance, the computational explorations of hyperboles remain under-explored. |
| Approach: | They propose a sentence-level hyperbole generation method that leverages commonsense and counterfactual inference to generate hyperbolic candidates based on the results. |
| Outcome: | The proposed method generates hyperboles with high success rate, intensity, funniness, and creativity. |
Socially Aware Bias Measurements for Hindi Language Representations (2022.naacl-main)
Copied to clipboard
| Challenge: | Language representations are an efficient tool used across NLP, but they are strife with encoded societal biases. |
| Approach: | They investigate the encoded biases in Hindi language representations based on cultural and historical contexts . they emphasize the necessity of social-awareness along with linguistic and grammatical artefacts when modeling language representation . |
| Outcome: | The proposed model reflects the cultural and cultural diversity of the region in which it is used . the model is based on the language and culture of the language being used based upon the study . |
Synchronous Faithfulness Monitoring for Trustworthy Retrieval-Augmented Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that RALMs generate baseless information or contradicts with the retrieved context. |
| Approach: | They propose a lightweight monitor that leverages fine-grained decoding dynamics to synchronously detect unfaithful sentences. |
| Outcome: | Empirical results show that SynCheck outperforms baseline faithfulness detection and FOD outperformed traditional strategies in terms of faithfulness. |
The Woman Worked as a Babysitter: On Biases in Language Generation (D19-1)
Copied to clipboard
| Challenge: | a systematic study of biases in natural language generation (NLG) is presented . a study of language models in NLG is conducted by examining language models. |
| Approach: | They propose a systematic study of biases in natural language generation by analyzing text generated from prompts that contain mentions of different demographic groups. |
| Outcome: | The proposed method reveals biases in natural language generation (NLG) by analyzing text generated from demographic prompts. |
Stack-Pointer Networks for Dependency Parsing (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to dependency parsing are local and greedy transitionbased . StackPtr parsers use the information of whole sentences and previously derived subtree structures . |
| Approach: | They propose a stack-pointer network-based dependency parser that reads whole sentence and builds dependency tree top-down in a depth-first fashion. |
| Outcome: | The proposed model reads and encodes whole sentence, then builds dependency tree top-down (from root-to-leaf) in a depth-first fashion. |
Scalable Construction and Reasoning of Massive Knowledge Bases (N18-6)
Copied to clipboard
| Challenge: | Existing knowledge mining systems assume abundant human annotations for training high quality machine learning models, which is impractical when trying to deploy IE systems to a broad range of domains, settings and languages. |
| Approach: | They introduce how to extract structured facts from text corpora to construct knowledge bases. |
| Outcome: | The proposed methods are weakly-supervised and domain-independent for knowledge base construction across various domains. |
Are Fairy Tales Fair? Analyzing Gender Bias in Temporal Narrative Event Chains of Children’s Fairy Tales (2023.acl-long)
Copied to clipboard
| Challenge: | Social biases and stereotypes are embedded in our culture through their presence in our stories. |
| Approach: | They propose a computational pipeline that automatically extracts a story’s temporal narrative verb-based event chain for each of its characters as well as character attributes such as gender. |
| Outcome: | The proposed framework extracts a story’s verb-based event chain for each of its characters as well as character attributes such as gender. |
DOC: Improving Long Story Coherence With Detailed Outline Control (2023.acl-long)
Copied to clipboard
| Challenge: | Detailed Outline Control (DOC) framework improves long-range plot coherence . human evaluations of DOC show it outperforms strong Re3 on plot cohesion, outline relevance and interestingness . |
| Approach: | They propose a Detailed Outline Control framework to improve long-range plot coherence . the detailed outliner creates a more detailed, hierarchically structured outline . they propose doc with a detailed controller to ensure the more detailed outline is respected . |
| Outcome: | The proposed framework outperforms Re3 on plot coherence, outline relevance and interestingness. |
Context-Situated Pun Generation (2022.emnlp-main)
Copied to clipboard
Jiao Sun, Anjali Narayan-Chen, Shereen Oraby, Shuyang Gao, Tagyoung Chung, Jing Huang, Yang Liu, Nanyun Peng
| Challenge: | a new task for context-situated pun generation uses a given context to generate puns . human evaluation shows that 69% of top retrieved pun words can be used to generate context-based puns. |
| Approach: | They propose a task where puns are generated based on contextual keywords and pun words. |
| Outcome: | The proposed system generates successful puns 31% of the time given a plausible tuple of context words and pun pairs. |
Vulnerability of Large Language Models to Output Prefix Jailbreaks: Impact of Positions on Safety (2025.findings-naacl)
Copied to clipboard
| Challenge: | Previous research on jailbreak attacks has focused on optimizing the adversarial snippet content injected into input prompts to expose LLM security vulnerabilities. |
| Approach: | They propose to use a simple adversarial snippet at the beginning of output to expose LLM security vulnerabilities. |
| Outcome: | The proposed approach exposes LLM security vulnerabilities much faster than input suffix attacks or prompt-based output jailbreaks. |
Character-centric Story Visualization via Visual Planning and Token Alignment (2022.emnlp-main)
Copied to clipboard
| Challenge: | Story visualization is a task that requires machines to understand long text inputs and produce a globally consistent image sequence that illustrates the contents of the story. |
| Approach: | They propose to augment VQ-VAE with a text-to-visual-token (transformer) architecture to enable multiple image generation based on a complete story. |
| Outcome: | The proposed method excels at preserving characters and produces higher quality image sequences compared with baselines. |
Improving Event Definition Following For Zero-Shot Event Detection (2024.acl-long)
Copied to clipboard
Zefan Cai, Po-Nien Kung, Ashima Suvarna, Mingyu Ma, Hritik Bansal, Baobao Chang, P. Jeffrey Brantingham, Wei Wang, Nanyun Peng
| Challenge: | Existing approaches on zero-shot event detection train models on datasets annotated with known event types and prompt them with unseen event definitions. |
| Approach: | They propose to train models to better follow event definitions by using an automatic generated Diverse Event Definition dataset. |
| Outcome: | The proposed model outperforms existing models on three open benchmarks on zero-shot event detection. |
DICE: Data-Efficient Clinical Event Extraction with Generative Models (2023.acl-long)
Copied to clipboard
| Challenge: | EE tasks target specific domains with vague entity boundaries, resulting in a lack of training data. |
| Approach: | They propose a robust and data-efficient generative model for clinical event extraction . they frame event extraction as a conditional generation problem and introduce a contrastive learning objective to decide the boundaries of biomedical mentions. |
| Outcome: | The proposed model is robust and data-efficient for clinical event extraction . it trains an auxiliary mention identification task and event extraction tasks to better identify entity mention boundaries . |
Model Extrapolation Expedites Alignment (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to improve LLM alignment training require expensive computational resources. |
| Approach: | They propose a model extrapolation method to expedite LLMs’ alignment with human preferences by amplifying parameter changes based on a first-order approximation without any additional training overhead. |
| Outcome: | The proposed method outperforms a fully-trained model on leading benchmarks and significantly outperformed open-source models. |
Societal Biases in Language Generation: Progress and Challenges (2021.acl-long)
Copied to clipboard
| Challenge: | Language generation techniques can produce undesirable societal biases that can negatively impact marginalized populations. |
| Approach: | They propose to examine how decoding techniques contribute to biases in language generation . they also conduct experiments to quantify the effects of these techniques . |
| Outcome: | The proposed methods can reduce biases and improve user experience, the authors argue . they also show that the proposed techniques can reduce societal biase . |