Papers by Xiang Ren
Agentic Knowledgeable Self-awareness (2025.acl-long)
Copied to clipboard
Shuofei Qiao, Zhisong Qiu, Baochang Ren, Xiaobin Wang, Xiangyuan Ru, Ningyu Zhang, Xiang Chen, Yong Jiang, Pengjun Xie, Fei Huang, Huajun Chen
| Challenge: | Large Language Models (LLMs) have achieved considerable performance across various agentic planning tasks. |
| Approach: | They propose a data-centric approach that applies agents with knowledgeable self-awareness like humans to a heuristic situation judgement criterion to mark special tokens on their self-explored trajectories for collecting training data. |
| Outcome: | The proposed paradigm outperforms baseline models on various tasks with minimal external knowledge. |
Reflect, Not Reflex: Inference-Based Common Ground Improves Dialogue Response Quality (2022.emnlp-main)
Copied to clipboard
| Challenge: | Currently, human communication models fail to explicitly model common ground (CG) . less than half of the responses in current data is rated as high quality . |
| Approach: | They propose a dataset that annotates dialogues with explicit CG and solicits 9k diverse responses each following one common ground. |
| Outcome: | The proposed dataset annotates dialogues with explicit CG and solicits 9k diverse responses each following one common ground. |
Probing Commonsense Explanation in Dialogue Response Generation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, response generation (RG) models do not understand human communication intents. |
| Approach: | They propose to examine commonsense reasoning implicitly to determine whether RG models produce coherent responses in conversations. |
| Outcome: | The proposed probing settings show that RG models fail to capture the logical relations between commonsense explanations and responses and fine-tuning on in-domain data do not lead to understanding of CSR for RG. |
I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragons (2023.acl-long)
Copied to clipboard
Pei Zhou, Andrew Zhu, Jennifer Hu, Jay Pujara, Xiang Ren, Chris Callison-Burch, Yejin Choi, Prithviraj Ammanabrolu
| Challenge: | Existing dialogue agents, while able to produce human-like responses, often do not model goal-driven and grounded language interactions. |
| Approach: | They propose to decompose and model teacher-student natural language interactions into (1) the DM’s intent to guide players toward a given goal; (2) the dm’s guidance utterance to the players expressing this intent; (3) a theory-of-mind model that anticipates the players’ reaction to the guidance one turn into the future. |
| Outcome: | The proposed task is based on a goal-driven and grounded environment with a teacher-student interaction model and theory-of-mind model. |
RockNER: A Simple Method to Create Adversarial Examples for Evaluating the Robustness of Named Entity Recognition Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent named entity recognition models have great performance on many conventional benchmarks, but it is not reliable in realistic applications. |
| Approach: | They propose a method to create natural adversarial examples using Wikidata and pre-trained language models. |
| Outcome: | The proposed method produces natural adversarial examples with a shifted distribution from training data. |
Instruction-following Evaluation through Verbalizer Manipulation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing benchmarks focus on common instructions that align well with what the model learned during training, but proficiency in responding to these instructions does not necessarily imply strong ability in instruction following. |
| Approach: | They propose a new instruction-following evaluation protocol called verbalizer manipulation that instructs the model to verbalize the task label with words aligning with model priors to different extents. |
| Outcome: | The proposed protocol can be integrated with any classification benchmark to examine the model’s reliance on priors and its ability to override them to accurately follow the instructions. |
Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuning (2023.emnlp-main)
Copied to clipboard
Ximing Lu, Faeze Brahman, Peter West, Jaehun Jung, Khyathi Chandu, Abhilasha Ravichander, Prithviraj Ammanabrolu, Liwei Jiang, Sahana Ramnath, Nouha Dziri, Jillian Fisher, Bill Lin, Skyler Hallinan, Lianhui Qin, Xiang Ren, Sean Welleck, Yejin Choi
| Challenge: | Extreme-scale language models have shown exceptional performance on a variety of language tasks, but the degree of control offered by these models through pure prompting is limited. |
| Approach: | They propose an inference-time policy adapter which tailors a large base model without fine-tuning it. |
| Outcome: | The proposed model outperforms baseline methods on five challenging text generation tasks and even over GPT-4. |
Facet-Aware Evaluation for Extractive Summarization (2020.acl-main)
Copied to clipboard
| Challenge: | lexical overlap is a common evaluation metric for extractive summarization, but recent studies reveal its limitations. |
| Approach: | They propose a facet-aware evaluation setup for better assessment of information coverage in extractive summaries. |
| Outcome: | The proposed evaluation setup improves human correlation with extractive summarization datasets and improves comparative analysis. |
Good Examples Make A Faster Learner: Simple Demonstration-based Learning for Low-resource NER (2022.acl-long)
Copied to clipboard
Dong-Ho Lee, Akshen Kadakia, Kangmin Tan, Mahak Agarwal, Xinyu Feng, Takashi Shibuya, Ryosuke Mitani, Toshiyuki Sekiya, Jay Pujara, Xiang Ren
| Challenge: | Recent advances in prompt-based learning have shown strong results on few-shot text classification by using cloze-style templates. |
| Approach: | They propose a demonstration-based learning method which lets the input be prefaced by task demonstrations for in-context learning. |
| Outcome: | The proposed method improves on in-domain learning and domain adaptation in low-resource settings. |
Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answering (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing QA systems do not have commonsense knowledge or cannot reason with it. |
| Approach: | They propose to augment a general commonsense QA framework with a knowledgeable path generator by extrapolating existing paths from a KG with 'state-of-the-art' language model. |
| Outcome: | The generated paths are interpretable, novel, and relevant to the task. |
On the Robustness of Reading Comprehension Models to Entity Renaming (2022.naacl-main)
Copied to clipboard
| Challenge: | SpanBERT model is more robust than RoBERTa, despite having similar accuracy on unperturbed test data. |
| Approach: | They propose a pipeline to replace entity names with names from a variety of sources. |
| Outcome: | The proposed model performs worse when entities are renamed, the authors show . SpanBERT, which is pretrained with span-level masking, is more robust than RoBERTa . |
RobustLR: A Diagnostic Benchmark for Evaluating Logical Robustness of Deductive Reasoners (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that perform deductive reasoning on inputs containing rules and statements in the English natural language do not perform consistently on the RobustLR test set. |
| Approach: | They propose a diagnostic benchmark that evaluates the robustness of language models to minimal logical edits in inputs and different logical equivalence conditions. |
| Outcome: | The proposed models do not perform consistently on the RobustLR test set. |
CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained language models perform well on commonsense-reasoning benchmark datasets, but building machines with commonsence to compose plausible sentences remains challenging. |
| Approach: | They propose a constrained text generation task for generative commonsense reasoning that generates a coherent sentence using common concepts. |
| Outcome: | The proposed task generates a coherent sentence describing an everyday scenario using common concepts over 35k concept-sets. |
KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning (D19-1)
Copied to clipboard
| Challenge: | empowering machines with the ability to perform commonsense reasoning has been seen as the bottleneck of artificial general intelligence . |
| Approach: | They propose a textual inference framework that uses external commonsense knowledge graphs to answer commonsensical questions. |
| Outcome: | The proposed framework is based on graph convolutional networks and LSTMs with a hierarchical path-based attention mechanism. |
Diagnosing Memorization in Chain-of-Thought Reasoning, One Token at a Time (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) perform well on reasoning benchmarks but often fail when inputs alter slightly, raising concerns about overreliance on memorization. |
| Approach: | They propose a framework for Source-aware Token-level Identification of Memorization which attributes each token in a reasoning chain to one of multiple memorization sources based on their statistical co-occurrence with the token in the pretraining corpus. |
| Outcome: | The proposed framework attributes each token in a reasoning chain to one of multiple memorization sources based on their statistical co-occurrence with the token in the pretraining corpus. |
Are Machines Better at Complex Reasoning? Unveiling Human-Machine Inference Gaps in Entailment Verification (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing evidence that humans make numerous inferences to understand discourse and text is not fully understood. |
| Approach: | They propose to use textual inference datasets with multi-sentence premises to solve the entailment verification problem. |
| Outcome: | The proposed model outperforms GPT-3.5 and rivals GPL-4 in EV tasks. |
Cross-lingual Continual Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing multi-lingual representations such as the one-hop transfer learning pipeline are difficult to adapt to new languages. |
| Approach: | They propose a cross-lingual continuum learning paradigm that evaluates continuous learning approaches that adapt to emerging data from different languages. |
| Outcome: | The proposed model can be used to adapt to new languages in a sequential manner. |
KCAT: A Knowledge-Constraint Typing Annotation Tool (P19-3)
Copied to clipboard
Sheng Lin, Luye Zheng, Bo Chen, Siliang Tang, Zhigang Chen, Guoping Hu, Yueting Zhuang, Fei Wu, Xiang Ren
| Challenge: | Recent years Natural Language Processing community has seen a surge of interest in fine-grained entity typing (FET) given an entity mention (i.e. a sequence of token spans representing an entity), FET aims at uncovering its contextdependent type. |
| Approach: | They propose an efficient Knowledge Constraint Fine-grained Entity Typing Annotation Tool which further improves the entity typing process through entity linking together with some practical functions. |
| Outcome: | The proposed tool improves the entity typing process by linking the candidate types with some practical functions. |
Self-Supervised Prompt Optimization (2025.findings-emnlp)
Copied to clipboard
Jinyu Xiang, Jiayi Zhang, Zhaoyang Yu, Xinbing Liang, Fengwei Teng, Jinhao Tu, Fashen Ren, Xiangru Tang, Sirui Hong, Chenglin Wu, Yuyu Luo
| Challenge: | Existing prompt optimization methods rely heavily on external references such as ground truth or by humans, limiting their applicability in real-world scenarios where such data is unavailable or costly to obtain. |
| Approach: | They propose a cost-efficient framework that discovers effective prompts for both closed and open-ended tasks without external reference. |
| Outcome: | The proposed framework outperforms state-of-the-art prompt optimization methods with significantly lower costs and fewer samples. |
FaiRR: Faithful and Robust Deductive Reasoning over Natural Language (2022.acl-long)
Copied to clipboard
| Challenge: | Currently, black-box models generate both the proof graph and intermediate inferences within the same model and thus may be unfaithful. |
| Approach: | They propose a transformer-based model that can perform deductive reasoning on a logical rulebase containing rules and statements written in natural language. |
| Outcome: | The proposed model is robust to language perturbations and faster at inference than previous models on existing reasoning datasets. |
Stepwise Informativeness Search for Improving LLM Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models have improved multistep reasoning but they lose focus over the middle of long contexts. |
| Approach: | They propose a tree search framework that proactively identifies underutilized steps and minimizing redundant information between steps. |
| Outcome: | The proposed framework generates more accurate and concise rationales with reduced errors and redundancy. |
On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning (2021.naacl-main)
Copied to clipboard
| Challenge: | PTLMs can exhibit biases against protected groups in a host of modeling tasks . but, fine-tuned LMs may propagate bias to downstream classifiers . |
| Approach: | They propose to use upstream bias mitigation techniques to reduce bias on downstream tasks by fine-tuning an upstream model and applying it to a downstream model. |
| Outcome: | The proposed model reduces bias on hate speech detection, toxicity detection and coreference resolution tasks over bias factors. |
Are Machine Rationales (Not) Useful to Humans? Measuring and Improving Human Utility of Free-text Rationales (2023.acl-long)
Copied to clipboard
Brihi Joshi, Ziyi Liu, Sahana Ramnath, Aaron Chan, Zhewei Tong, Shaoliang Nie, Qifan Wang, Yejin Choi, Xiang Ren
| Challenge: | Existing metrics like task performance of the LM generating the rationales or similarity between generated and gold rationale are not good indicators of their human utility. |
| Approach: | They propose to use a large language model to generate rationales with better human utility by estimating its conciseness and novelty. |
| Outcome: | The proposed model can measure human utility to a better extent by estimating its usefulness in answering similar unseen instances. |
Efficient Contextualized Representation: Language Model Pruning for Sequence Labeling (D18-1)
Copied to clipboard
| Challenge: | Existing efforts to train pre-trained language models have brought significant improvements to various NLP applications. |
| Approach: | They propose to compress bulky LMs while preserving useful information for a specific task. |
| Outcome: | The proposed method can detach any layer without affecting others, and stretch shallow and wide LMs to be deep and narrow. |
LLM-driven Instruction Following: Progresses and Concerns (2023.emnlp-tutorial)
Copied to clipboard
| Challenge: | a tutorial on task instruction is aimed at researchers and practitioners interested in NLP generalization . labeled examples are unlikely to be available in large numbers or do not exist . |
| Approach: | This tutorial will examine the progress of natural language processing (NLP) using labeled examples. authors propose that task instructions act as a novel resource for supervision. |
| Outcome: | This tutorial aims to answer questions about instruction-driven NLP . it focuses on the use of task instructions in a low-shot scenario . |
RiddleSense: Reasoning about Riddle Questions Featuring Linguistic Creativity and Commonsense Knowledge (2021.findings-acl)
Copied to clipboard
| Challenge: | a riddle-style commonsense questions require complex commonsensense reasoning and figurative language skills . there is currently no dataset aimed at testing these abilities . authors propose a new multiple-choice question answering task . |
| Approach: | They propose a new multiple-choice question answering task that uses a large dataset for riddlestyle commonsense questions. |
| Outcome: | The proposed task comes with the first large dataset for answering riddlestyle commonsense questions. |
LEAN-LIFE: A Label-Efficient Annotation Framework Towards Learning from Explanation (2020.acl-demos)
Copied to clipboard
Dong-Ho Lee, Rahul Khanna, Bill Yuchen Lin, Seyeon Lee, Qinyuan Ye, Elizabeth Boschee, Leonardo Neves, Xiang Ren
| Challenge: | Existing frameworks for sequence labeling and classification require massive human effort and labeling data is limited. |
| Approach: | They propose a web-based, Label-Efficient AnnotatioN framework that allows an annotator to provide the needed labels for a task and can capture explanations for each labeling decision. |
| Outcome: | The proposed framework surpasses baseline F1 scores by 5-10 percentage points while using 2X times fewer labeled instances. |
Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models that pursue rapid generalization to new tasks are mostly trained in a single shot on fixed datasets, unable to dynamically expand their knowledge. |
| Approach: | They propose a new learning setup that assumes a model learns from a sequence of diverse NLP tasks arriving sequentially, accumulating knowledge for improved generalization to new tasks. |
| Outcome: | The proposed learning setup improves generalization ability while retaining performance on the tasks learned earlier. |
Differentiable Open-Ended Commonsense Reasoning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing commonsense reasoning models work by scoring a question-candidate pair, but new approaches are needed to answer multiple-choice questions. |
| Approach: | They propose to use a corpus of commonsense facts to answer a commonsensical question without any pre-defined choices as a resource. |
| Outcome: | The proposed model outperforms baseline methods by a large margin in the open-ended commonsense reasoning task. |
Machine Translation Robustness to Natural Asemantic Variation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing machine translation models struggle with noisy data and tail-end words and phrases. |
| Approach: | They introduce and formalize a class of noise and variation that preserves meaning in the target language. |
| Outcome: | The proposed model can perform better on natural asemantic variation (NAV) the proposed model is robust to a variety of perturbations, but not all of them are achieved with organic variations. |
ECONET: Effective Continual Pretraining of Language Models for Event Temporal Reasoning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models (PTLMs) have achieved noticeable success on many NLP tasks, but struggle for tasks that require event temporal reasoning. |
| Approach: | They propose a continual pre-training approach that equips PTLMs with targeted knowledge about event temporal relations by focusing on masked-out event and temporal indicators and discriminating sentences from their corrupted counterparts. |
| Outcome: | The proposed framework improves the PTLMs’ fine-tuning performances across five relation extraction and question answering tasks and achieves new or on-par state-of-the-art in most of our downstream tasks. |
Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection (2024.naacl-long)
Copied to clipboard
Jun Yan, Vikas Yadav, Shiyang Li, Lichang Chen, Zheng Tang, Hai Wang, Vijay Srinivasan, Xiang Ren, Hongxia Jin
| Challenge: | Instruction-tuned Large Language Models (LLMs) can modulate responses based on human instructions, but they can be maliciously steered to impact society in subtle but persistent ways. |
| Approach: | They propose a backdoor attack setting that allows an attacker to inject a virtual prompt into an LLM to steer it without any explicit injection at its input. |
| Outcome: | The proposed method is able to poison the model's instruction tuning data and show that it is highly effective in steering the model. |
Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on augmenting question answering models with external knowledge (e.g., knowledge graphs) lacks transparency into the model’s prediction rationale. |
| Approach: | They propose a knowledge-aware approach that equips pre-trained language models with a multi-hop relational reasoning module that performs multi-relational reasoning over subgraphs extracted from external knowledge graphs. |
| Outcome: | The proposed model performs multi-hop, multi-relational reasoning over subgraphs extracted from external knowledge graphs. |
TriggerNER: Learning with Entity Triggers as Explanations for Named Entity Recognition (2020.acl-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a fundamental information extraction task that focuses on extracting entities from a given text and classifying them using pre-defined categories. |
| Approach: | They propose to use “entity triggers” to facilitate label-efficient learning of NER models. |
| Outcome: | The proposed model is significantly more cost-effective than the traditional neural NER frameworks. |
Leveraging Visual Knowledge in Language Tasks: An Empirical Study on Intermediate Pre-training for Cross-Modal Knowledge Transfer (2022.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models lack visual knowledge of common objects due to reporting bias. |
| Approach: | They investigate whether integrating visual knowledge into a language model can fill the gap . they use captions and images to transfer visual knowledge to 5 downstream tasks . |
| Outcome: | The proposed model can improve performance on 5 tasks that may need visual knowledge to solve the problem. |
Discretized Integrated Gradients for Explaining Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Integrated Gradients (IG) is widely adopted due to its desirable explanation axioms and the ease of gradient computation. |
| Approach: | They propose an attribution-based explanation algorithm that uses averaging the model's output gradient interpolated along a straight-line path in the input data space. |
| Outcome: | The proposed method is compared with IG on multiple sentiment classification datasets. |
Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | a series of experiments show that fine-tuning only the cross-attention parameters is nearly as effective as fine-timing all parameters. |
| Approach: | They conduct experiments to fine-tune a translation model on data where either the source or target language has changed. |
| Outcome: | The proposed model can be trained to several new languages with reduced parameter storage overhead. |
Hierarchical Text Classification with Reinforced Label Assignment (D19-1)
Copied to clipboard
| Challenge: | Existing hierarchical text classification methods make local decisions regarding labels or ignore hierarchy information during inference. |
| Approach: | They propose to learn a Label Assignment Policy via deep reinforcement learning to determine where to place an object and when to stop the assignment process. |
| Outcome: | The proposed method outperforms state-of-the-art methods on five datasets and four base models and achieves an average improvement of 33.4% over flat classifiers. |
APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoning (2023.acl-long)
Copied to clipboard
Soumya Sanyal, Yichong Xu, Shuohang Wang, Ziyi Yang, Reid Pryzant, Wenhao Yu, Chenguang Zhu, Xiang Ren
| Challenge: | Existing methods to improve logical reasoning skills require complex data processing. |
| Approach: | They propose an adaptive pretraining approach to improve logical reasoning over text . they use a subset of Wikipedia sentences for pretraining and a sentence-level classification loss . |
| Outcome: | The proposed model outperforms baselines on LogiQA and ReClor. |
SynSetExpan: An Iterative Framework for Joint Entity Set Expansion and Synonym Discovery (2020.emnlp-main)
Copied to clipboard
| Challenge: | Entity set expansion and synonym discovery are two critical NLP tasks that are often performed separately, without exploring their interdependencies. |
| Approach: | They propose a framework that enables two tasks to mutually enhance each other by including popular entities’ infrequent synonyms into the set, which boosts set expansion recall. |
| Outcome: | The proposed framework can be used to enhance two NLP tasks by including popular entities’ infrequent synonyms into the set, which boosts set expansion recall. |
WildVis: Open Source Visualizer for Million-Scale Chat Logs in the Wild (2024.emnlp-demo)
Copied to clipboard
| Challenge: | Currently, the volume and complexity of chat logs makes it difficult to analyze individual conversations. |
| Approach: | They propose a tool that enables fast, versatile, and large-scale conversation analysis by combining search and visualization capabilities with a list of criteria. |
| Outcome: | The proposed tool can be extended to handle millions of chat logs and other datasets. |
A Good Prompt Is Worth Millions of Parameters: Low-resource Prompt-based Learning for Vision-Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Recent few-shot learning models such as GPT3 are expensive and slow to deploy for real-world applications. |
| Approach: | They propose a prompt-based low-resource learning method for VL tasks with a few examples . they pre-train a sequence-to-sequence transformer model with prefix and masked language modeling . |
| Outcome: | The proposed method outperforms Frozen on vision-language tasks with prompt-based learning by 18.2% point. |
Estimating Large Language Model Capabilities without Labeled Test Data (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models have shown impressive ability to perform in-context learning from only a few examples, but their accuracy varies widely from task to task. |
| Approach: | They propose a method that trains a meta-model using LLM confidence scores as features to perform ICL accuracy estimation. |
| Outcome: | The proposed method improves over baselines across 7 out of 12 settings and achieves the same accuracy as evaluating on 40 sampled examples per task. |
FiD-ICL: A Fusion-in-Decoder Approach for Efficient In-Context Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Large pre-trained models are capable of few-shot in-context learning (ICL) however, concatenated demonstrations are often excessively long and require additional computation. |
| Approach: | They propose to apply fusion-in-decoder (FiD) models to perform few-shot in-context learning (ICL) they propose to use concatenation-based, early-fusion, intermediate- and late-fusion methods to improve efficiency . |
| Outcome: | The proposed methods outperform concatenation-based models on 11 held-out tasks. |
Believing without Seeing: Quality Scores for Contextualizing Vision-Language Model Explanations (2026.acl-long)
Copied to clipboard
| Challenge: | Prior work has found that explanations can easily convince users that inaccurate VLM predictions are correct. |
| Approach: | They propose to evaluate two complementary qualities of VLM-generated explanations via two quality scoring functions to improve their accuracy. |
| Outcome: | The proposed explanations improve accuracy on the A-OKVQA, VizWiz, and MMMU-Pro tasks by 11.1%, including a 15.4% reduction in falsely believing incorrect predictions. |
Sparse Distillation: Speeding Up Text Classification by Using Bigger Student Models (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to reduce inference cost by distilling transformer models into lightweight student models are limited for high-volume use cases. |
| Approach: | They propose to distill state-of-the-art transformer models into lightweight student models to reduce computation cost at inference time. |
| Outcome: | The proposed pipeline achieves up to 600x speed-up on GPUs and CPUs on six single-sentence text classification tasks and in domain generalization settings. |
Teaching Machine Comprehension with Compositional Explanations (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in machine reading comprehension rely heavily on large-scale annotated corpora, which are timeconsuming and costly to collect. |
| Approach: | They propose to use semi-structured explanations to “teach” machines reading comprehension using a small number of semi-structural explanations that explicitly inform machines why answer spans are correct. |
| Outcome: | The proposed method achieves 70.14% F1 score with supervision from 26 explanations on the SQuAD dataset, comparable to plain supervised learning using 1,100 labeled instances yielding a 12x speed up. |
Eliciting Knowledge from Experts: Automatic Transcript Parsing for Cognitive Task Analysis (P19-1)
Copied to clipboard
| Challenge: | Cognitive task analysis (CTA) is a type of analysis used to elicit and represent the knowledge and thought processes of domain experts. |
| Approach: | They propose a weakly-supervised framework for automated CTA transcript parsing . they partition the parser process into a sequence labeling task and a text span-pair relation extraction task with distant supervision from human-curated protocol files. |
| Outcome: | The proposed framework reduces human labor and scales the task to a small scale. |
Recurrent Event Network: Autoregressive Structure Inferenceover Temporal Knowledge Graphs (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for reasoning over temporal knowledge graphs focus on past timestamps and are not able to predict future interactions. |
| Approach: | They propose a novel autoregressive architecture for predicting future interactions using a recurrent event encoder and a neighborhood aggregator. |
| Outcome: | The proposed method achieves state-of-the-art on five public datasets. |
Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-Trained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained language models possess certain commonsense and factual knowledge. |
| Approach: | They propose to use pre-trained language models to predict masked words . they introduce a probing task with 13.6k m-word-prediction probes . |
| Outcome: | The proposed model performs poorly on the diagnostic dataset prior to any fine-tuning and fine-testing with distant supervision. |
Rethinking Backdoor Detection Evaluation for Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing backdoor detection methods have high accuracy in detecting backdoored models, but they are not robust enough to detect backdoors in the wild. |
| Approach: | They examine the robustness of backdoor detectors by manipulating different factors during backdoor planting. |
| Outcome: | The proposed methods are able to detect backdoors in the wild, but they lack robustness against backdoor attacks. |
Amulet: Putting Complex Multi-Turn Conversations on the Stand with LLM Juries (2025.emnlp-main)
Copied to clipboard
| Challenge: | a typical human-assistant conversation is lengthy and shows significant diversity in topics, intents, and requirements across turns. |
| Approach: | They propose a framework that leverages pertinent linguistic concepts of dialog-acts and maxims to improve the accuracy of LLM-judges on preference data with complex, multi-turn conversational context. |
| Outcome: | The proposed framework improves on 4 challenging datasets showing that humans frequently change their intents from one turn of the conversation to the next. |
Learning to Generate Task-Specific Adapters from Task Description (2021.acl-short)
Copied to clipboard
| Challenge: | Pre-trained text-to-text transformers have achieved impressive performance across a range of NLP tasks, such as question answering and commonsense reasoning. |
| Approach: | They propose a framework that improves text-to-text transformer’s generalization ability to unseen tasks by training a hypernetwork to generate task-specific adapters from task descriptions. |
| Outcome: | Experiments on ZEST and a synthetic SQuAD dataset show that Hypter improves upon fine-tuning baselines. |
Improving Distantly-supervised Entity Typing with Compact Latent Space Clustering (N19-1)
Copied to clipboard
| Challenge: | Existing studies have addressed this problem with partial-label loss, but they suffer from confirmation bias, which means the classifier fit a pseudo data distribution given by itself. |
| Approach: | They propose to regularize distantly supervised models with Compact Latent Space Clustering to bypass this problem and effectively utilize noisy data yet. |
| Outcome: | The proposed model outperforms state-of-the-art models on standard benchmarks on fine-grained entity typing (FET) by a significant margin. |
NewsEdits: A News Article Revision Dataset and a Novel Document-Level Reasoning Challenge (2022.naacl-main)
Copied to clipboard
| Challenge: | a large dataset of news article revision histories provides clues to narrative and factual evolution in news articles. |
| Approach: | They propose tasks to predict edit-actions performed during version updates . they define article-level edit actions: Addition, Deletion, Edit and Refactor . |
| Outcome: | The proposed dataset is large-scale and multilingual and spans 15 years . it shows that edit-actions are predictable and are likely to be based on factual evolution . |
Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoning (2021.acl-long)
Copied to clipboard
| Challenge: | Using multilingual language models, commonsense reasoning research has been limited to English. |
| Approach: | They propose a Mickey Probe task to evaluate commonsense across languages . they propose X-CSQA and XCODAH datasets to be translated to 14 languages based on the Mickey corpus . |
| Outcome: | The proposed method significantly improves sentence representations beyond English. |
Contextualizing Hate Speech Classifiers with Post-hoc Explanation (2020.acl-main)
Copied to clipboard
| Challenge: | Modern text classifiers struggle to learn a model of hate speech that generalizes to real-world applications. |
| Approach: | They propose a method to regularize BERT classifiers to detect bias towards identity terms by providing explanations for group identifiers and allowing models to learn from the context of group identifiers. |
| Outcome: | The proposed method limiting false positives on out-of-domain data while maintaining and improving in-domain performance. |
Relying on the Unreliable: The Impact of Language Models’ Reluctance to Express Uncertainty (2024.acl-long)
Copied to clipboard
| Challenge: | a pivotal aspect of fostering reliable human-AI interactions lies in the apt communication of model confidences. |
| Approach: | They examine how LMs incorporate confidence in responses via natural language . they also examine how downstream users behave in response to LM-articulated uncertainties . |
| Outcome: | The proposed model overconfidences are high in LMs, and humans are biased against uncertainty-rich texts. |
Think Before You Speak: Explicitly Generating Implicit Commonsense Knowledge for Response Generation (2022.acl-long)
Copied to clipboard
Pei Zhou, Karthik Gopalakrishnan, Behnam Hedayatnia, Seokhwan Kim, Jay Pujara, Xiang Ren, Yang Liu, Dilek Hakkani-Tur
| Challenge: | Current neural response generation models generate responses directly, omitting unstated implicit knowledge. |
| Approach: | They propose a generative approach to externalize implicit commonsense knowledge and use it to generate responses. |
| Outcome: | Empirical results show that TBS models outperform end-to-end RG models on most automatic metrics and generate more informative, specific, and commonsense-following responses. |
Lawyers are Dishonest? Quantifying Representational Harms in Commonsense Knowledge Resources (2021.emnlp-main)
Copied to clipboard
| Challenge: | Commonsense knowledge bases are mostly human-generated and reflect societal biases . a filtering-based approach can reduce the issues in both resources and models but leads to a performance drop . |
| Approach: | They propose a filtering-based approach to mitigating representational harms in ConceptNet and GenericsKB . they propose filtered-based approaches can reduce issues in both resources and models but leads to performance drop . |
| Outcome: | The proposed approach reduces issues in resources and models but leads to performance drop . the paper proposes a filtering-based approach that reduces biases but leaves room for future work . |
CAVE: Controllable Authorship Verification Explanations (2025.naacl-long)
Copied to clipboard
| Challenge: | Authorship Verification (AV) is used for tasks such as plagiarism detection, forensic analysis, analysis of the spread of misinformation. |
| Approach: | They propose to train an offline authorship verification model that is accessible and easy to use. |
| Outcome: | The proposed model generates high quality explanations and competitive task accuracy on three difficult AV datasets. |
Multi-document Summarization with Maximal Marginal Relevance-guided Reinforcement Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies on single-document summarization (SDS) benefit from advances in neural sequence learning, but they produce unsatisfactory results on multi-document summary (MDS). |
| Approach: | They propose a neural sequence learning method that unifies advanced neural SDS methods and statistical measures used in classical MDS. |
| Outcome: | The proposed method achieves state-of-the-art performance on benchmark MDS datasets. |
BITE: Textual Backdoor Attacks with Iterative Trigger Injection (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to defend against backdoor attacks are based on model stealing, model thieving and training data extraction attacks. |
| Approach: | They propose a backdoor attack that poisons training data to establish strong correlations between the target label and a set of “trigger words” These trigger words are iteratively identified and injected into the target-label instances through natural word-level perturbations. |
| Outcome: | The proposed attack is significantly more effective than baseline methods while maintaining decent stealthiness, raising alarm on the usage of untrusted training data. |
Looking Beyond Label Noise: Shifted Label Distribution Matters in Distantly Supervised Relation Extraction (D19-1)
Copied to clipboard
| Challenge: | Existing studies on DS-based relation extraction (RE) methods focus on handling label noise, but other factors may have been overlooked. |
| Approach: | They propose a method to automatically adjust DS-RE models to a shifted label distribution problem . they find this problem exists in real-world DS datasets and can be overcome . |
| Outcome: | The proposed method achieves consistent performance gains on DS-trained models with an up to 23% relative F1 improvement, which verifies their assumptions. |
Collaborative Policy Learning for Open Knowledge Graph Reasoning (D19-1)
Copied to clipboard
| Challenge: | Existing models of knowledge graph reasoning suffer from limited performance when working on sparse and incomplete graphs due to the lack of evidential paths that can reach target entities. |
| Approach: | They propose a framework to train two collaborative agents to reason for missing facts over a graph augmented by a text corpus. |
| Outcome: | Experiments on two public datasets show the proposed approach is effective on a knowledge graph reasoning task. |
CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP (2021.emnlp-main)
Copied to clipboard
| Challenge: | We study whether and how cross-task generalization ability can be acquired . we use CrossFit to standardize seen/unseen task partitions and evaluation protocols . |
| Approach: | They propose a problem setup for studying cross-task generalization ability which standardizes seen/unseen task partitions and data access during different learning stages. |
| Outcome: | The proposed model can be used to build few-shot learners across diverse tasks. |
Learning Contextualized Knowledge Structures for Commonsense Reasoning (2021.findings-acl)
Copied to clipboard
Jun Yan, Mrigank Raman, Aaron Chan, Tianyu Zhang, Ryan Rossi, Handong Zhao, Sungchul Kim, Nedim Lipka, Xiang Ren
| Challenge: | Recent knowledge graph (KG) augmented models have achieved notable success on commonsense reasoning tasks. |
| Approach: | They propose a KG-augmented model that contextualizes extracted and generated knowledge by reasoning over both within a single graph structure. |
| Outcome: | The proposed model outperforms existing models on four commonsense reasoning benchmarks and a user study on edge validness and helpfulness. |
Symbolic Working Memory Enhances Language Models for Complex Rule Application (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) excel in single-step rule application but struggle with multi-step deductive reasoning when rules are presented non-sequentially. |
| Approach: | They propose to augment LLMs with external working memory and introduce a neurosymbolic framework for rule application that stores facts and rules in both natural language and symbolic forms, enabling precise tracking. |
| Outcome: | The proposed framework iteratively performs symbolic rule grounding and LLM-based rule implementation. |
Do Language Models Perform Generalizable Commonsense Inference? (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent work has applied pretrained language models to populate commonsense knowledge graphs (CKGs) but there is a lack of understanding on their generalization to multiple CKGs, unseen relations, and novel entities. |
| Approach: | They analyze the ability of pretrained language models to perform generalizable commonsense inference in terms of knowledge capacity, transferability and induction. |
| Outcome: | The proposed models can adapt to different schemas defined by multiple CKGs but fail to generalize to new relations. |
Exploring Distributional Shifts in Large Language Models for Code Analysis (2023.emnlp-main)
Copied to clipboard
| Challenge: | Since the late 2000s, researchers have been reporting poor generalization of statistical learning models to new software systems, such as GitHub Copilot, Amazon CodeWhisperer, Replit, etc. |
| Approach: | They systematically study how three large language models with code capabilities generalize to out-of-domain data. |
| Outcome: | The proposed model outperforms the existing model for code generation on multiple domains at once. |
AutoTriggER: Label-Efficient and Robust Named Entity Recognition with Auxiliary Trigger Extraction (2023.eacl-main)
Copied to clipboard
Dong-Ho Lee, Ravi Kiran Selvam, Sheikh Muhammad Sarwar, Bill Yuchen Lin, Fred Morstatter, Jay Pujara, Elizabeth Boschee, James Allan, Xiang Ren
| Challenge: | Named entity recognition models have shown impressive results in overcoming label scarcity and generalizing to unseen entities by leveraging distant supervision and auxiliary information such as explanations. |
| Approach: | They propose a framework that automatically generates and leverages “entity triggers” which are human-readable cues in the text that help guide the model to make better decisions. |
| Outcome: | The proposed framework outperforms the RoBERTa-CRF baseline by nearly 0.5 F1 points on three well-studied datasets. |
MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application (2026.acl-long)
Copied to clipboard
Xueqing Peng, Lingfei Qian, Yan Wang, Ruoyu Xiang, Yueru He, Yang Ren, Mingyang Jiang, Vincent Jim Zhang, Yuqing Guo, Jeff Zhao, Huan He, Yi Han, Yun Feng, Yuechen Jiang, Yupeng Cao, Haohang Li, Yangyang Yu, Xiaoyu Wang, Penglei Gao, Shengyuan Lin, Keyi Wang, Shanshan Yang, Yilun Zhao, Zhiwei Liu, Peng Lu, Jerry Huang, Suyuchen Wang, Triantafillos Papadopoulos, Polydoros Giannouris, Efstathia Soufleri, Nuo Chen, Zhiyang Deng, Heming Fu, Yijia Zhao, Mingquan Lin, Meikang Qiu, Kaleb E Smith, Arman Cohan, Xiao-Yang Liu, Jimin Huang, Guojun Xiong, Alejandro Lopez-Lira, Xi Chen, Junichi Tsujii, Jian-Yun Nie, Sophia Ananiadou, Qianqian Xie
| Challenge: | Existing evaluations of LLMs in finance are text-only, monolingual, and largely saturated by current models. |
| Approach: | They propose a multilingual and multimodal benchmark for evaluating LLMs in real financial contexts. |
| Outcome: | The first expert-annotated multilingual and multimodal benchmark is released . it evaluates 21 leading LLMs and shows they perform better in multilingual settings . |
FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Tasks (2022.findings-naacl)
Copied to clipboard
Bill Yuchen Lin, Chaoyang He, Zihang Ze, Hulin Wang, Yufen Hua, Christophe Dupuy, Rahul Gupta, Mahdi Soltanolkotabi, Xiang Ren, Salman Avestimehr
| Challenge: | Increasing concerns and regulations about data privacy necessitate the study of privacy-preserving, decentralized learning methods for natural language processing tasks. |
| Approach: | They propose a framework for evaluating federated learning methods on four different tasks . they propose federation between Transformer-based language models and FL methods . |
| Outcome: | The proposed framework compares FL methods on four different tasks under non-IID partitioning strategies. |
ELI-Why: Evaluating the Pedagogical Utility of Language Model Explanations (2025.findings-acl)
Copied to clipboard
Brihi Joshi, Keyu He, Sahana Ramnath, Sadra Sabouri, Kaitlyn Zhou, Souti Chattopadhyay, Swabha Swayamdipta, Xiang Ren
| Challenge: | Language models are widely used in education, yet their ability to tailor responses to learners with varied informational needs and knowledge backgrounds remains under-explored. |
| Approach: | They conduct two extensive human studies to assess the utility of language model-generated explanatory answers (explanations) on a benchmark of 13.4K "Why" questions. |
| Outcome: | The proposed model explanations match learners' educational backgrounds only 50% of the time, compared to 79% for lay explanations. |
LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion (2023.acl-long)
Copied to clipboard
| Challenge: | a recent study shows that open-source large language models (LLMs) exhibit diverse strengths and weaknesses due to variations in their architectures and training data. |
| Approach: | They propose a framework that leverages the diverse strengths of open-source large language models. |
| Outcome: | The proposed framework outperforms individual LLMs and baseline methods across various metrics, establishing a substantial performance gap. |
Knowledge-Augmented Methods for Natural Language Processing (2022.acl-tutorials)
Copied to clipboard
| Challenge: | Knowledge in natural language processing (NLP) is a rising trend especially after the advent of large scale pre-trained models. |
| Approach: | This tutorial introduces the key steps in integrating knowledge into natural language processing (NLP) it introduces knowledge grounding from text, knowledge representation and fusing. |
| Outcome: | This tutorial introduces the key steps in integrating knowledge into natural language processing including knowledge grounding from text, knowledge representation and fusing. |
UR2 : Unify RAG and Reasoning through Reinforcement Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing attempts to unify large language models are limited to open-domain QA with fixed retrieval settings. |
| Approach: | They propose a general reinforcement learning framework that dynamically coordinates retrieval and reasoning. |
| Outcome: | The proposed framework outperforms existing paradigms on open-domain QA, MMLU-Pro, medical, and mathematical reasoning tasks. |
TaxoClass: Hierarchical Multi-Label Text Classification Using Only Class Names (2021.naacl-main)
Copied to clipboard
| Challenge: | Hierarchical multi-label text classification (HMTC) aims to assign each text document to a set of relevant classes from a taxonomy. |
| Approach: | They propose to conduct HMTC based on only class surface names as supervision signals to mimic human experts. |
| Outcome: | The proposed framework outperforms the best existing method by 25% on two challenging datasets. |
Learning Collaborative Agents with Rule Guidance for Knowledge Graph Reasoning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Walk-based models have shown their advantages in knowledge graph reasoning but are limited by their representations and generalizability. |
| Approach: | They propose a walk-based model that leverages high-quality rules generated by symbolic-based methods to provide reward supervision for walk- based agents. |
| Outcome: | Experiments on benchmark datasets show that RuleGuider improves the performance of walk-based models without losing interpretability. |
Chain-Talker: Chain Understanding and Rendering for Empathetic Conversational Speech Synthesis (2025.findings-acl)
Copied to clipboard
| Challenge: | Current generative CSS models face interpretability limitations due to insufficient emotional perception and redundant discrete speech coding. |
| Approach: | They propose a framework that aligns synthesized speech with the emotional context of user-agent interactions to achieve empathy. |
| Outcome: | The proposed framework produces more expressive speech than existing methods on three datasets. |
REV: Information-Theoretic Evaluation of Free-Text Rationales (2023.acl-long)
Copied to clipboard
| Challenge: | Existing metrics for rationale evaluation focus on the association between the rationale and a label, whereas REV is more sensitive to new information in free-text rationales. |
| Approach: | They propose a metric called REV to quantify the amount of new, label-relevant information in a rationale beyond the information already available in the input or the label. |
| Outcome: | The proposed metric is consistent with human judgments on rationale evaluations and provides more sensitive measurements of new information in free-text rationales. |
MSD: Saliency-aware Knowledge Distillation for Multimodal Understanding (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Current knowledge distillation models are limited and lack performance on multimodal datasets. |
| Approach: | They propose a multimodal knowledge distillation framework to transfer knowledge from a teacher on multimodal tasks by learning the teacher's behavior within each modality. |
| Outcome: | The proposed framework achieves better performance than KD on four multimodal datasets. |
Disambiguation in Conversational Question Answering in the Era of LLMs and Agents: A Survey (2025.emnlp-main)
Copied to clipboard
Mehrab Tanjim, Yeonjun In, Xiang Chen, Victor Bursztyn, Ryan A. Rossi, Sungchul Kim, Guang-Jie Ren, Vaishnavi Muppala, Shun Jiang, Yongsung Kim, Chanyoung Park
| Challenge: | Existing literature on ambiguity and disambiguation with Large Language Models (LLMs) ambiguities are a fundamental challenge in human-AI interactions due to complexity and flexibility of human language. |
| Approach: | They propose to define key terms and concepts and categorize various disambiguation approaches enabled by LLMs and provide a comparative analysis of their advantages and disadvantages. |
| Outcome: | The proposed frameworks are compared against different disambiguation approaches and highlight their relevance for future research. |
On the Influence of Masking Policies in Intermediate Pre-training (2021.emnlp-main)
Copied to clipboard
Qinyuan Ye, Belinda Z. Li, Sinong Wang, Benjamin Bolte, Hao Ma, Wen-tau Yih, Xiang Ren, Madian Khabsa
| Challenge: | Existing studies show that inserting an intermediate pre-training stage improves performance of masked language models. |
| Approach: | They propose methods to automate the discovery of optimal masking policies via direct supervision or meta-learning. |
| Outcome: | The proposed method outperforms the heuristic of masking named entities on TriviaQA and can be generalizable beyond that task. |
Symbolic Chain-of-Thought Distillation: Small Models Can Also “Think” Step-by-Step (2023.acl-long)
Copied to clipboard
| Challenge: | Symbolic Chain-of-thought Distillation (SCoTD) is a method to train a smaller student model on rationalizations sampled from a significantly larger teacher model. |
| Approach: | They propose a method to train a smaller student model on rationalizations from a larger teacher model. |
| Outcome: | The proposed method improves the performance of a student model in supervised and few-shot settings and especially for challenge sets. |
Will This Idea Spread Beyond Academia? Understanding Knowledge Transfer of Scientific Concepts across Text Corpora (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research on knowledge transfer focuses on documents as unit of analysis and follow their transfer into practice for a specific scientific domain. |
| Approach: | They analyze scientific concepts from corpora and use them to predict knowledge transfer . they find that only a small proportion of these ideas will be used in inventions . |
| Outcome: | The proposed model predicts the use of scientific concepts in clinical trials and inventions. |
XMD: An End-to-End Framework for Interactive Explanation-Based Debugging of NLP Models (2023.acl-demo)
Copied to clipboard
Dong-Ho Lee, Akshen Kadakia, Brihi Joshi, Aaron Chan, Ziyi Liu, Kiran Narahari, Takashi Shibuya, Ryosuke Mitani, Toshiyuki Sekiya, Jay Pujara, Xiang Ren
| Challenge: | Existing models are susceptible to learning spurious biases that do not reflect the underlying task. |
| Approach: | They propose an open-source framework for explanation-based model debugging that allows users to provide various forms of feedback on model explanations. |
| Outcome: | The proposed framework improves model’s OOD performance on text classification tasks by up to 18%. |
Visually Grounded Continual Learning of Compositional Phrases (2020.emnlp-main)
Copied to clipboard
| Challenge: | Modern NLP systems rely on offline training and are inefficient for new tasks. |
| Approach: | They propose a visually grounded ContinuaL learning task which simulates the continual acquisition of compositional phrases from streaming visual scenes. |
| Outcome: | The proposed system improves on existing systems, but it's infeasible to store all possible compositions. |
HMEAE: Hierarchical Modular Event Argument Extraction (D19-1)
Copied to clipboard
| Challenge: | Existing event extraction methods classify each argument role independently, ignoring conceptual correlations between different argument roles. |
| Approach: | They propose a Hierarchical Modular Event Argument Extraction model to provide inductive bias from the concept hierarchy of event argument roles. |
| Outcome: | The proposed model outperforms existing methods on real-world datasets and shows that it leverages useful knowledge from the concept hierarchy. |
On Continual Model Refinement in Out-of-Distribution Data Streams (2022.acl-long)
Copied to clipboard
| Challenge: | Existing continual learning (CL) problems cannot cover real-world scenarios such as out-of-distribution errors. |
| Approach: | They propose a continual model refinement problem formulation to solve this problem . they extend several existing continual learning approaches to the CMR problem based on a general sampling algorithm . |
| Outcome: | The proposed model refinement solution improves on existing models and their performance metrics. |
AlpacaTag: An Active Learning-based Crowd Annotation Framework for Sequence Tagging (P19-3)
Copied to clipboard
| Challenge: | Existing sequence annotation tools focus on improving user interfaces and user interface. |
| Approach: | They propose an open-source web-based data annotation framework for sequence tagging tasks . the framework is based on active learning and automatic crowd consolidation . |
| Outcome: | The proposed framework is a comprehensive solution for sequence labeling tasks . it can be deployed in downstream systems while new annotations are being made . |
How Predictable Are Large Language Model Capabilities? A Case Study on BIG-bench (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that large language models can be used to predict performance on new configurations. |
| Approach: | They investigate the predictability of large language model capabilities by using BIG-bench . they find a subset of BIG-Bench tasks as informative as BIG-bnch Hard . |
| Outcome: | The proposed model achieves an R2 score greater than 95% on BIG-bench . the model is 3 smaller than BIG-Bench Hard, and the model performs better on the full set. |
Extract, Denoise and Enforce: Evaluating and Improving Concept Preservation for Text-to-Text Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for text-to-text generation do not explicitly focus on important concepts in the input and output. |
| Approach: | They propose a framework to automatically extract, denoise, and enforce important input concepts as lexical constraints. |
| Outcome: | The proposed framework performs comparably or better than its unconstrained counterpart on automatic metrics and receives better ratings in the human evaluation. |
CLAPSpeech: Learning Prosody from Text Context with Contrastive Language-Audio Pre-Training (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for expressive text-to-speech only implicitly learn prosody with masked token reconstruction tasks. |
| Approach: | They propose a cross-modal contrastive pre-training framework that learns from prosody variance of the same text token under different contexts. |
| Outcome: | The proposed framework can learn from prosody variance of a text token under different contexts. |
Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora (2022.naacl-main)
Copied to clipboard
| Challenge: | Pretrained language models are typically learned over a large, static corpus and fine-tuned for various downstream tasks. |
| Approach: | They propose to continuously update a pretrained language model to adapt to emerging data and to keep track of the model's performance. |
| Outcome: | The proposed model can adapt to new corpora while retaining knowledge in earlier domains. |
Mixing Inference-time Experts for Enhancing LLM Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for improving reasoning quality in large language models are limited to using a single expert. |
| Approach: | They propose a framework that finetunes and merges expert logits from one LLM . they use commonsense and entailment reasoning experts to improve chain-of-thought reasoning . |
| Outcome: | The proposed framework outperforms baselines on three question-answering datasets. |
Learning Named Entity Tagger using Domain-Specific Dictionary (D18-1)
Copied to clipboard
| Challenge: | Existing methods to build reliable named entity recognition systems require large amounts of manually-labeled training data. |
| Approach: | They propose a revised fuzzy CRF layer to handle tokens with multiple possible labels to address noisy distant supervision. |
| Outcome: | The proposed model can handle tokens with multiple possible labels under the traditional framework and improves on the existing model with a new Tie or Break scheme. |
A Semantic Uncertainty Sampling Strategy for Back-Translation in Low-Resources Neural Machine Translation (2025.acl-srw)
Copied to clipboard
Yepai Jia, Yatu Ji, Xiang Xue, Shilei@imufe.edu.cn Shilei@imufe.edu.cn, Qing-Dao-Er-Ji Ren, Nier Wu, Na Liu, Chen Zhao, Fu Liu
| Challenge: | Back-translation methods rely on large-scale parallel corpora to enhance performance, but ignore the semantic quality of monolingual data. |
| Approach: | They propose a method which prioritizes sentences with higher semantic uncertainty as training samples by computationally evaluating the complexity of unannotated monolingual data. |
| Outcome: | The proposed method improves translation accuracy and fluency by +1.7 on all three translation tasks. |
AdaTag: Multi-Attribute Value Extraction from Product Profiles with Adaptive Decoding (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to extract product attribute values are limited by knowledge sharing across different attributes. |
| Approach: | They propose to use adaptive decoding to handle extraction of product attribute values by parameterizing the decoder with pretrained attribute embeddings, through a hypernetwork and a Mixture-of-Experts module. |
| Outcome: | The proposed model is able to handle multiple attributes without sharing the entire network parameters across all attributes. |
End-to-End Reinforcement Learning for Automatic Taxonomy Induction (P18-1)
Copied to clipboard
| Challenge: | Existing methods for automating taxonomy induction often divide the problem into two subtasks . a novel end-to-end reinforcement learning approach is proposed to improve the accuracy of such methods. |
| Approach: | They propose an end-to-end reinforcement learning approach to automatic taxonomy induction from a set of terms. |
| Outcome: | The proposed approach outperforms state-of-the-art methods on two public datasets of different domains. |
RICA: Evaluating Robust Inference Capabilities Based on Commonsense Axioms (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models have impressive performance on commonsense inference benchmarks, but their ability to make robust inferences is debated. |
| Approach: | They propose a challenge that evaluates robust commonsense inference despite textual perturbations using commonsensical knowledge bases and probe PTLMs across two different evaluation settings. |
| Outcome: | The proposed procedure evaluates robust commonsense inference despite textual perturbations using commonsensense knowledge bases and probe PTLMs across two evaluation settings. |
ER-Test: Evaluating Explanation Regularization Methods for Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Explanation regularization (ER) aims to improve NLM generalization by pushing the NLM’s machine rationales to align with human rationale. |
| Approach: | They propose a framework for evaluating ER models’ OOD generalization along three dimensions: unseen datasets, contrast set tests, and functional tests. |
| Outcome: | The proposed framework evaluates ER models’ OOD generalization across unseen datasets, contrast set tests, and functional tests. |
SCOTT: Self-Consistent Chain-of-Thought Distillation (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models (LMs) generate free-text rationales for their predictions via chain-of-thought prompting, but there is little guarantee that the generated rationale is consistent with LM’s predictions or faithfully justify the decisions. |
| Approach: | They propose a faithful knowledge distillation method to learn a small, self-consistent CoT model from a larger teacher model by contrastive decoding. |
| Outcome: | The proposed method yields comparable performance but is less faithful than baselines. |
Learning Dynamic Context Augmentation for Global Entity Linking (D19-1)
Copied to clipboard
Xiyuan Yang, Xiaotao Gu, Sheng Lin, Siliang Tang, Yueting Zhuang, Fei Wu, Zhigang Chen, Guoping Hu, Xiang Ren
| Challenge: | Existing collective entity linking methods are expensive and often lack local context information. |
| Approach: | They propose a dynamic context-augmented inference model that can be used to make collective inference. |
| Outcome: | The proposed model can cope with different local EL models with different learning settings, base models, decision orders and attention mechanisms. |
AV-TranSpeech: Audio-Visual Robust Speech-to-Speech Translation (2023.acl-long)
Copied to clipboard
Rongjie Huang, Huadai Liu, Xize Cheng, Yi Ren, Linjun Li, Zhenhui Ye, Jinzheng He, Lichao Zhang, Jinglin Liu, Xiang Yin, Zhou Zhao
| Challenge: | Existing models for speech-to-speech translation suffer from distinct degradation in noisy environments and fail to translate visual speech. |
| Approach: | They propose a text-based audio-visual speech-to-speech translation model that integrates visual information with audio-only data to improve system robustness. |
| Outcome: | The proposed model outperforms models trained on audio-only corpus in two languages . it also improves with low-resource audio-visual data, compared with baselines . |
DRInQ: Evaluating Conversational Implicature with Controlled Context Variation (2026.acl-long)
Copied to clipboard
| Challenge: | Recent large language models exhibit strong conversational fluency but are unreliable when interpretation depends on reasoning that integrates social and contextual cues. |
| Approach: | They propose a semi-automated pipeline that produces question-context-interpretation instances with systematic variation to isolate pragmatic variation while holding each question’s surface form fixed. |
| Outcome: | The proposed framework isolates pragmatic variation while holding each question’s surface form fixed. |
Posterior-regularized REINFORCE for Instance Selection in Distant Supervision (N19-1)
Copied to clipboard
| Challenge: | Existing methods to train unbiased methods such as REINFORCE take time to train. |
| Approach: | They propose to use posterior regularization to integrate domain-specific rules in instance selection using REINFORCE to improve the performance of the relation classifier trained on cleaned distant supervision datasets. |
| Outcome: | The proposed method improves the performance of the relation classifier trained on cleaned distant supervision dataset as well as the efficiency of the REINFORCE training. |
Improving Language Model Personas via Rationalization with Psychological Scaffolds (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to building personas rely on a user’s demographic attributes and/or prior judgments, but not on any underlying reasoning behind a person’s judgments. |
| Approach: | They propose a framework that integrates rationales for why a user could have made a certain judgment into LM personas by incorporating potential rationale. |
| Outcome: | The proposed framework outperforms models conditioned on demographic attributes and/or prior judgments on public opinion and movie preference prediction tasks. |
In Search of the Long-Tail: Systematic Generation of Long-Tail Inferential Knowledge via Logical Rule Guided Search (2024.emnlp-main)
Copied to clipboard
Huihan Li, Yuting Ning, Zeyi Liao, Siyuan Wang, Xiang Li, Ximing Lu, Wenting Zhao, Faeze Brahman, Yejin Choi, Xiang Ren
| Challenge: | Logic-Induced-Knowledge-Search (LINK) is a framework for generating factually-correct yet long-tail inferential knowledge. |
| Approach: | They introduce a framework to obtain factually-correct yet long-tail inferential statements using variable-wise prompting grounded on symbolic rules. |
| Outcome: | The proposed framework is able to obtain factually-correct yet long-tail inferential statements while ensuring factual correctness. |
Scalable Construction and Reasoning of Massive Knowledge Bases (N18-6)
Copied to clipboard
| Challenge: | Existing knowledge mining systems assume abundant human annotations for training high quality machine learning models, which is impractical when trying to deploy IE systems to a broad range of domains, settings and languages. |
| Approach: | They introduce how to extract structured facts from text corpora to construct knowledge bases. |
| Outcome: | The proposed methods are weakly-supervised and domain-independent for knowledge base construction across various domains. |
KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering (2022.acl-long)
Copied to clipboard
Donghan Yu, Chenguang Zhu, Yuwei Fang, Wenhao Yu, Shuohang Wang, Yichong Xu, Xiang Ren, Yiming Yang, Michael Zeng
| Challenge: | Open-Domain Question Answering (ODQA) models typically include a retrieving module and a reading module. |
| Approach: | They propose a new open-domain question-answering framework that uses a knowledge-enhanced version of FiD to improve the approach. |
| Outcome: | The proposed model improves on ODQA benchmark datasets with less than 40% computation cost. |
REL-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing evaluations of large language models' ability to communicate uncertainty and knowledge limitations focus on the behaviors of their human interlocutors. |
| Approach: | They propose an interaction-centered evaluation approach that quantifies whether and how humans rely on LLMs' responses. |
| Outcome: | The proposed approach quantifies whether and how humans rely on LLMs' responses. |
Contrastive Novelty-Augmented Learning: Anticipating Outliers with Large Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for classification are overly confident on unseen examples . despite recent advances in NLP, some categories of distribution shift still pose serious challenges. |
| Approach: | They propose a method that generates OOD examples representative of novel classes and trains to decrease confidence on them. |
| Outcome: | The proposed method improves classifiers' ability to detect and abstain on novel class examples over previous methods by 2.3% and 5.5% over previous approaches. |
Can LLMs Reason with Rules? Logic Scaffolding for Stress-Testing and Improving LLMs (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive human-like performance across various reasoning tasks, but their mastery of underlying inferential rules falls short of human capabilities. |
| Approach: | They propose a logic scaffolding inferential rule generation framework to construct an infer- ential rule base, ULogic, comprising both primitive and compositional rules across five domains. |
| Outcome: | The proposed model improves the ability to generate accurate, complex and abstract conclusions and premises and improves various commonsense reasoning tasks. |
Learning to Contextually Aggregate Multi-Source Supervision for Sequence Labeling (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for Sequence Labeling require high-quality annotations, but imperfect annotations are relatively easy to obtain from crowdsourcing (noisy labels) Existing approaches to learn a model without knowing the underlying ground truth label sequences in the target domain are expensive and time-consuming. |
| Approach: | They propose a framework Consensus Network that can be trained on annotations from multiple sources. |
| Outcome: | The proposed framework improves on learning with crowd annotations and unsupervised cross-domain model adaptation in two practical settings. |
Reporting the Unreported: Event Extraction for Analyzing the Local Representation of Hate Crimes (D19-1)
Copied to clipboard
Aida Mostafazadeh Davani, Leigh Yeh, Mohammad Atari, Brendan Kennedy, Gwenyth Portillo Wightman, Elaine Gonzalez, Natalie Delong, Rhea Bhatia, Arineh Mirinjian, Xiang Ren, Morteza Dehghani
| Challenge: | Existing estimates of hate crimes in the US are under-reported relative to actual number of incidents. |
| Approach: | They propose to use event extraction and multi-instance learning to predict hate crimes in local news articles for cities without official FBI reports. |
| Outcome: | The proposed model compares to FBI reports and shows that hate crimes are under-reported in local press. |
ForecastQA: A Question Answering Challenge for Event Forecasting with Temporal Text Data (2021.acl-long)
Copied to clipboard
| Challenge: | Existing automated forecasting studies rely on structured data to predict future events. |
| Approach: | They propose a question-answering task that limits access to unstructured text data . they use a crowdsourced dataset to form a restricted-domain, multiple-choice, question-announcement task . |
| Outcome: | The proposed model achieves 61.0% accuracy on the dataset, which still lags behind human performance by about 19%. |
Eliciting and Understanding Cross-task Skills with Task-level Mixture-of-Experts (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained transformer models are capable of multitasking on diverse NLP tasks, but little is known about how multitaskability and cross-task generalization is achieved. |
| Approach: | They propose to use a transformer-based mixture-of-expert model with a router component to choose among experts dynamically and flexibly. |
| Outcome: | The proposed models improve the average performance gain (ARG) metric by 2.6% when adapting to unseen tasks, and by 5.6% in zero-shot generalization settings. |
X-METRA-ADA: Cross-lingual Meta-Transfer learning Adaptation to Natural Language Understanding and Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Multilingual models have gained popularity for their zero-shot cross-lingual transfer learning capabilities, but their generalization ability is inconsistent for typologically diverse languages. |
| Approach: | They propose a meta-learning approach that adapts MAML to learn to adapt to new languages . they extensively evaluate two cross-lingual NLU tasks using English as source and spanish as target . |
| Outcome: | The proposed approach outperforms naive fine-tuning on cross-lingual tasks for most languages. |