Papers by Alex Chen
Enabling Self-Improving Agents to Learn at Test Time With Human-In-The-Loop Guidance (2025.emnlp-industry)
Copied to clipboard
Yufei He, Ruoyu Li, Alex Chen, Yue Liu, Yulin Chen, Yuan Sui, Cheng Chen, Yi Zhu, Luca Luo, Frank Yang, Bryan Hooi
| Challenge: | Existing large language model (LLM) agents are unable to adapt to changing domain knowledge and rules. |
| Approach: | They propose an LLM agent framework that continuously learns updated domain knowledge at test time. |
| Outcome: | The proposed agent improves on a customer due diligence name screening task on . the agent learns updated domain knowledge at test time. |
Label Representations in Modeling Classification as Text Generation (2020.aacl-srw)
Copied to clipboard
| Challenge: | Existing methods for text generation use strings to represent labels . linguistic properties of labels do affect performance, though their results are limited to document retrieval. |
| Approach: | They investigate the effect of string representations on how effectively a model learns a task . they use four standard text classification tasks to model string representation . |
| Outcome: | The proposed model improves on four standard text classification tasks . the results are largely negative in the low data setting . |
The Pitfalls of KV Cache Compression (2026.acl-long)
Copied to clipboard
| Challenge: | Recent literature has shown minimal degradation of KV cache in multi-instruction prompts . authors show that certain instructions degrade much more rapidly with compression . |
| Approach: | They propose to change KV cache eviction policies to reduce the impact of KV evict bias . they propose to use a 'simple' evviction policy to reduce ejection bias if the LLM is a multi-instruction model . |
| Outcome: | The proposed methods show that certain instructions degrade much faster with compression, causing them to be ignored by the LLM. |
What Do NLP Researchers Believe? Results of the NLP Community Metasurvey (2023.acl-long)
Copied to clipboard
Julian Michael, Ari Holtzman, Alicia Parrish, Aaron Mueller, Alex Wang, Angelica Chen, Divyam Madaan, Nikita Nangia, Richard Yuanzhe Pang, Jason Phang, Samuel R. Bowman
| Challenge: | Getting sociological beliefs wrong can slow research and lead to wasted effort, missed opportunities, and needless fights. |
| Approach: | They present the results of the NLP Community Metasurvey, run from May to June 2022. |
| Outcome: | The NLP community metasurvey elicited opinions on controversial issues from May to June 2022. |
Abstract Meaning Representation-Based Logic-Driven Data Augmentation for Logical Reasoning (2024.findings-acl)
Copied to clipboard
Qiming Bao, Alex Peng, Zhenyun Deng, Wanjun Zhong, Gael Gendron, Timothy Pistotti, Neset Tan, Nathan Young, Yang Chen, Yonghua Zhu, Paul Denny, Michael Witbrock, Jiamou Liu
| Challenge: | Empirical evidence shows that our proposed method improves performance across seven downstream tasks. |
| Approach: | They propose a logic-driven data augmentation approach that converts text into AMR graphs and converts them back into text to create augmented data. |
| Outcome: | The proposed method leads on the ReClor leaderboard and improves on seven downstream tasks. |
AfriInstruct: Instruction Tuning of African Languages for Diverse Tasks (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) for African languages perform worse compared to high-resource languages. |
| Approach: | They propose a model that specializes in instruction-tuning of multiple African languages covering various tasks. |
| Outcome: | The proposed model outperforms GPT-3.5-Turbo and other models of similar size in multiple tasks. |
SQuALITY: Building a Long-Document Summarization Dataset the Hard Way (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing summarization datasets often have issues that seriously limit their usability. |
| Approach: | They propose a faster but more straightforward approach to developing summarization benchmark data . they use a protocol that hires highly-qualified contractors to read stories and write original summaries from scratch . |
| Outcome: | The proposed protocol is faster but more straightforward than scraping summaries from everyday text. |
A Cheaper and Better Diffusion Language Model with Soft-Masked Noise (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing diffusion models have limitations in modeling discrete data, e.g., languages . we present a novel diffusion model for language modeling inspired by linguistic features in languages based on iterative denoising . |
| Approach: | They propose a method that iteratively denoises and adds corruptions to the textual data through soft-masking to better noise it. |
| Outcome: | The proposed model achieves better generation quality and lower training cost than current models with better performance. |
CAT-Gen: Improving Robustness in NLP Models via Controlled Adversarial Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing adversarial text generation approaches can lead to generation lacking diversity or fluency, whereas perturbing in the intermediate representation space can lead a model to generate generations that are not related to the input. |
| Approach: | They propose to generate adversarial texts through controllable attributes that are known to be invariant to task labels. |
| Outcome: | The proposed model generates more diverse and fluent adversarial examples, compared to existing approaches, and is more robust against model re-training and different model architectures. |
Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling (P19-1)
Copied to clipboard
Alex Wang, Jan Hula, Patrick Xia, Raghavendra Pappagari, R. Thomas McCoy, Roma Patel, Najoung Kim, Ian Tenney, Yinghui Huang, Katherin Yu, Shuning Jin, Berlin Chen, Benjamin Van Durme, Edouard Grave, Ellie Pavlick, Samuel R. Bowman
| Challenge: | State-of-the-art models in natural language processing (NLP) often incorporate sentence encoder functions which generate a sequence of vectors intended to represent the in-context meaning of each word in an input text. |
| Approach: | They conduct the first large-scale systematic study of candidate pretraining tasks, comparing 19 different tasks as alternatives and complements to language modeling. |
| Outcome: | The proposed model can be used to train sentences on language modeling tasks. |
Query-Key Normalization for Transformers (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Low-resource language translation is a challenging but socially valuable NLP task. |
| Approach: | They propose a normalization technique that modifies the attention mechanism to make the softmax function less prone to arbitrary saturation without sacrificing expressivity. |
| Outcome: | The proposed technique improves 0.928 BLEU over state-of-the-art bilingual benchmarks for 5 low-resource translation pairs from the TED Talks corpus and IWSLT’15. |
On Reference (In-)Determinacy in Natural Language Inference (2025.findings-naacl)
Copied to clipboard
Sihao Chen, Chaitanya Malaviya, Alex Fabrikant, Hagai Taitelbaum, Tal Schuster, Senaka Buthpitiya, Dan Roth
| Challenge: | Using reference determinacy, models fail to recognize context mismatch in NLI examples . |
| Approach: | They propose a benchmark to identify reference ambiguity in NLI examples . they propose RD as a possible assumption for natural language inference . |
| Outcome: | The proposed benchmark identifies reference ambiguity in natural language inference examples . 80% false contradiction and >50% entailment predictions are found . |
PropSegmEnt: A Large-Scale Corpus for Proposition-Level Segmentation and Entailment Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing systems for Natural Language Inference (NLI) only recognize textual entailment relations on sentence-level . however, even a simple sentence often contains multiple propositions, i.e. distinct units of meaning conveyed by the sentence . |
| Approach: | They propose a system to recognize whether one text is textually entailed by another . they use a corpus of over 45K propositions annotated by human raters to study the textual entailment relation of each proposition in a sentence individually. |
| Outcome: | The proposed dataset can be used to understand the compositionality of NLI labels. |
Language Agnostic Code Embeddings (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent studies show that code language models have strong cross-lingual traits, but their multilingual representations can be dissected into a language-specific syntax component and a semantic component. |
| Approach: | They propose to isolate and eliminate language-specific components from multilingual code embeddings to improve downstream code retrieval tasks. |
| Outcome: | The proposed model improves retrieval tasks by removing language-specific components . the proposed model can be used to perform a variety of code generation tasks . |
Improving Diversity of Demographic Representation in Large Language Models via Collective-Critiques and Self-Voting (2023.emnlp-main)
Copied to clipboard
Preethi Lahoti, Nicholas Blumm, Xiao Ma, Raghavendra Kotikalapudi, Sahitya Potluri, Qijun Tan, Hansa Srinivasan, Ben Packer, Ahmad Beirami, Alex Beutel, Jilin Chen
| Challenge: | Existing studies on diversity in large language models focus on the understudied class of fairness and inclusion concern in LLMs. |
| Approach: | They propose a technique to measure diversity in generated responses along people and culture axes by collective-critique and self-voting. |
| Outcome: | The proposed approach outperforms baseline methods and human evaluations with human and automated evaluations. |
Multi2Claim: Generating Scientific Claims from Multi-Choice Questions for Scientific Fact-Checking (2023.eacl-main)
Copied to clipboard
Neset Tan, Trung Nguyen, Josh Bensemann, Alex Peng, Qiming Bao, Yang Chen, Mark Gahegan, Michael Witbrock
| Challenge: | Existing scientific fact-checking datasets are limited due to expertise bottleneck . multi2Claim pipeline is a tool to convert multiple-choice questions into fact- checking data . |
| Approach: | They propose a pipeline for automatically converting multiple-choice questions into fact-checking data . they generate two large-scale datasets for scientific-fact-checker tasks . success at this task can help the reader understand scientific topics and promote science . |
| Outcome: | The proposed pipeline improves performance on two large-scale scientific fact-checking datasets. |
Few-shot In-context Learning on Knowledge Base Question Answering (2023.acl-long)
Copied to clipboard
| Challenge: | KB-BINDER enables few-shot in-context learning over knowledge base questions . KBQA is a difficult problem due to the heterogeneity of knowledge bases . |
| Approach: | They propose a framework that enables few-shot in-context learning over KBQA tasks. |
| Outcome: | The proposed framework can outperform state-of-the-art models on GraphQA and MetaQA datasets. |
DeCoRe: Decoding by Contrasting Retrieval Heads to Mitigate Hallucinations (2025.findings-emnlp)
Copied to clipboard
Aryo Pradipta Gema, Chen Jin, Ahmed Abdulaal, Tom Diethe, Philip Alexander Teare, Beatrice Alex, Pasquale Minervini, Amrutha Saseendran
| Challenge: | Large Language Models often produce unfaithful or factually incorrect outputs . masking retrieval heads can induce hallucinations, but decoding by contrast can reduce hallucinosity . |
| Approach: | They propose a training-free decoding strategy that contrasts the outputs of the base LLM and the masked LLM. |
| Outcome: | The proposed decoding strategy reduces hallucinations by contrasting the outputs of the base and masked LLMs. |
Improving Classifier Robustness through Active Generative Counterfactual Data Augmentation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for finding meaningful counterfactuals rely on human annotation or implicit label invariance . a small amount of human-annotated counterf actual data can generate a robust dataset with learned labels. |
| Approach: | They propose a framework that generates counterfactuals by actively sampling from regions of uncertainty and automatically labeling them with a learned auxiliary classifier. |
| Outcome: | The proposed framework generates a large number of diverse counterfactuals and labels them with a learned classifier. |
EquiBench: Benchmarking Large Language Models’ Reasoning about Program Semantics via Equivalence Checking (2025.emnlp-main)
Copied to clipboard
Anjiang Wei, Jiannan Cao, Ran Li, Hongyu Chen, Yuhui Zhang, Ziheng Wang, Yuan Liu, Thiago S. F. X. Teixeira, Diyi Yang, Ke Wang, Alex Aiken
| Challenge: | EquiBench is a new benchmark to evaluate large language models' ability to reason about program semantics . Unlike natural language, code is executable. |
| Approach: | They propose a benchmark to evaluate large language models through equivalence checking . EquiBench consists of 2400 program pairs across four languages and six categories . |
| Outcome: | The proposed benchmark consists of 2400 program pairs across four languages and six categories. |
Scaling Collaborative Effort with Agents (2026.findings-acl)
Copied to clipboard
Shannon Zejiang Shen, Valerie Chen, Ken Gu, Alexis Ross, Zixian Ma, Jillian Ross, Alex Gu, Chenglei Si, Wayne Chi, Andi Peng, Jocelyn J Shen, Ameet Talwalkar, Tongshuang Wu, David Sontag
| Challenge: | Current evaluations of agents focus on producing high-quality, final outputs in one shot, failing to account for the inherently iterative nature of many real-world problems. |
| Approach: | They propose a framework that captures how an agent’s utility grows with increasing user involvement. |
| Outcome: | The proposed framework captures how an agent’s utility grows with increasing user involvement, revealing a missing ingredient in agent design: the ability to sustain engagement and scaffold user understanding. |
Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusters (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in modeling and datasets demonstrate promising performance for NLI. |
| Approach: | They explore the direct zero-shot applicability of NLI models to real applications . they analyze the robustness of models to longer and out-of-domain inputs . |
| Outcome: | The proposed models are robust to longer and out-of-domain inputs and can perform on full documents. |