Papers by Ben Zhou

41 papers
Deceptive Semantic Shortcuts on Reasoning Chains: How Far Can Models Go without Hallucination? (2024.naacl-long)

Copied to clipboard

Challenge: Existing large language models (LLMs) suffer from hallucinations and unfaithful reasoning due to keyword/entity biases.
Approach: They propose a new probing method and benchmark to quantify this phenomenon by using a keyword/entity biases-based probing technique called EUREQA.
Outcome: The proposed method achieves 62% accuracy on multi-hop and complex QA benchmarks.
MSEarth: A Multimodal Benchmark for Earth Science Phenomenon Discovery with MLLMs (2026.acl-long)

Copied to clipboard

Challenge: Existing datasets often rely on synthetic data or figure-caption pairs, failing to capture the depth and complexity of geoscientific reasoning.
Approach: They propose a multimodal scientific dataset and benchmark curated from open-access publications.
Outcome: MSEarth features over 289K figures with captions enriched by contextual discussions and reasoning from original papers.
V-ALPHASOCIAL: Benchmark and Self-Reflective Chain-of-Thought Generation for Visual Social Commonsense Reasoning (2025.findings-acl)

Copied to clipboard

Challenge: Social commonsense reasoning is a multimodal task that requires both textual and visual cues.
Approach: They propose a method that integrates visual cues into social commonsense reasoning tasks.
Outcome: The proposed method improves social commonsense reasoning on a multimodal foundation model.
Navigating the Shadows: Unveiling Effective Disturbances for Modern AI Content Detectors (2024.acl-long)

Copied to clipboard

Challenge: Recent research indicates that AI-text detection systems lack robustness and struggle to effectively differentiate perturbed texts.
Approach: They propose to evaluate the robustness of current detection systems by using black-box text perturbation methods and adversarial learning experiments.
Outcome: The proposed methods assess the robustness of current detection models across perturbation granularities and the impact of perturbation data augmentation on the robustity of AI-text detectors.
Familiarity-Aware Evidence Compression for Retrieval-Augmented Generation (2025.findings-emnlp)

Copied to clipboard

Challenge: Retrieval-augmented generation (RAG) improves large language models by incorporating non-parametric knowledge through evidence retrieved from external sources.
Approach: They propose a training-free evidence compression technique that makes retrieved evidence more familiar to the target model while seamlessly integrating parametric knowledge from the model.
Outcome: The proposed technique outperforms the most recent evidence compression baselines across open-domain QA datasets while achieving high compression rates.
There’s a Time and Place for Reasoning Beyond the Image (2022.acl-long)

Copied to clipboard

Challenge: Currently, most work in this area is focused on reasoning with local evidence, but there is a gap between a state-of-the-art joint model and human performance.
Approach: They propose a model that can be used to infer, associate, and reason with contextual information from other sources to establish a more complete picture.
Outcome: The proposed model can identify the news articles related to the image through segment-wise understandings of the signs, buildings, crowds, and more.
“Going on a vacation” takes longer than “Going for a walk”: A Study of Temporal Commonsense Understanding (D19-1)

Copied to clipboard

Challenge: a new study examines temporal commonsense and compares it to human performance on a dataset . a previous study focused on duration, frequency, stationarity and ordering, but not all aspects of temporal similarity have been studied.
Approach: They define five classes of temporal commonsense and use crowdsourcing to develop a new dataset that serves as a test set.
Outcome: The proposed dataset shows that the best current methods are far behind human performance by 20%.
Sub-Sentence Encoder: Contrastive Learning of Propositional Semantic Representations (2024.naacl-long)

Copied to clipboard

Challenge: Sentence embeddings are typically learned to recognize the semantic relation between two text inputs.
Approach: They introduce a contrastively-learned contextual embedding model for fine-grained semantic representation of text.
Outcome: The proposed model is able to produce contextual embeddings corresponding to different atomic propositions, i.e. semantic equivalence between propositions across different text sequences.
Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking (2024.findings-naacl)

Copied to clipboard

Challenge: Large language models (LLMs) have demonstrated increasing power, but they also have vulnerabilities.
Approach: They propose a black-box attack that targets the cognitive structure and processes of large language models (LLMs) they propose defending cognitive overload attacks from three perspectives.
Outcome: The proposed attack is a black-box attack with no need for knowledge of model architecture or access to model weights.
DiffLM: Controllable Synthetic Data Generation via Diffusion Language Models (2025.findings-acl)

Copied to clipboard

Challenge: Recent advances in large language models (LLMs) have significantly enhanced their knowledge and generative capabilities, leading to a surge of interest in leveraging LLMs for high-quality data synthesis.
Approach: They propose a controllable data synthesis framework based on variational autoencoder which leverages diffusion models to reserve more information of original distribution and format structure in the learned latent distribution.
Outcome: The proposed framework generates high-quality data with performance exceeding that of real data by 2%–7% on seven real-world datasets.
Temporal Common Sense Acquisition with Minimal Supervision (2020.acl-main)

Copied to clipboard

Challenge: Temporal common sense is crucial for understanding natural language, but its acquisition is challenging . human annotation on such concepts is costly and often not made explicit in text .
Approach: They propose a method that exploits explicit and implicit mentions of temporal common sense to build a temporal similarity language model.
Outcome: The proposed model gives better predictions of various dimensions of temporal common sense than the standard BERT.
ConsistentChat: Building Skeleton-Guided Consistent Multi-Turn Dialogues for Large Language Models from Scratch (2025.emnlp-main)

Copied to clipboard

Challenge: Existing instruction data synthesis methods focus on single-turn instructions and neglect cross-turn coherence, resulting in context drift and reduced task completion rates.
Approach: They propose a framework that constrains multi-turn instruction synthesis by explicitly modeling human conversational intent.
Outcome: The proposed framework outperforms existing models trained on single-turn and multi-turn instruction datasets.
Reliable Use of Lemmas via Eligibility Reasoning and Section-Aware Reinforcement Learning (2026.acl-short)

Copied to clipboard

Challenge: Recent large language models (LLMs) perform strongly on mathematical benchmarks but often import conclusions without validating assumptions.
Approach: They propose a model that encodes a lemma specification and trains with reinforcement learning and section-aware loss masking to assign penalty to the section responsible for errors.
Outcome: The proposed model performs well on benchmarks but often misapplyes lemmas . the model is able to encode the specification and train with reinforcement learning .
Temporal Reasoning on Implicit Events from Distant Supervision (2021.naacl-main)

Copied to clipboard

Challenge: a novel temporal reasoning dataset evaluates the degree to which systems understand implicit events . state-of-the-art models struggle when predicting temporal relationships between implicit and explicit events - a recent paper .
Approach: They propose a temporal reasoning dataset that evaluates the degree to which systems understand implicit events.
Outcome: The proposed model outperforms baseline systems on TRACIE by 5% and 11% on MATRES, an explicit event benchmark.
RESIN: A Dockerized Schema-Guided Cross-document Cross-lingual Cross-media Information Extraction and Event Tracking System (2021.naacl-demos)

Copied to clipboard

Challenge: We present a new information extraction system that can construct temporal event graphs from news documents.
Approach: They propose a temporal event graph extraction system that can extract news documents . they extend the system from sentence-level event extraction to cross-document cross-media event extraction .
Outcome: The proposed system can extract temporal event graphs from news documents in multiple languages and multiple data modalities.
Enhancing LLM Capabilities Beyond Scaling Up (2024.emnlp-tutorials)

Copied to clipboard

Challenge: general-purpose large language models (LLMs) are expanding in scale and access to unpublic training data.
Approach: This tutorial aims to examine the capabilities of general-purpose large language models . authors discuss adaptation of LLMs to address conflicts, defense against attacks .
Outcome: This tutorial aims to examine the evolution of general-purpose large language models (LLMs) the authors argue that the evolution is dependent on the availability of training data and the scale of the models.
ResearchBench: Benchmarking LLMs in Scientific Discovery via Inspiration-Based Task Decomposition (2026.findings-acl)

Copied to clipboard

Challenge: Large language models have shown potential in assisting scientific research, yet their ability to discover high-quality research hypotheses remains unexamined due to the lack of a dedicated benchmark.
Approach: They propose a benchmark for evaluating large language models on a sufficient set of scientific discovery sub-tasks.
Outcome: The proposed framework extracts critical components from papers across 12 disciplines with expert validation confirming its accuracy.
ThinkTuning: Instilling Cognitive Reflections without Distillation (2025.emnlp-main)

Copied to clipboard

Challenge: Recent advances in test-time scaling have led to the emergence of thinking LLMs that exhibit self-reflective behaviors and multi-step reasoning.
Approach: They propose a GRPO-based interactive training approach that augments the rollouts of a student model with the guidance of . a teacher poses a problem, lets the student try an answer, then gives corrective feedback–enough to point the mind in the right direction and then show the correct solution.
Outcome: The proposed method shows 3.69% improvement over zero-shot baselines and 2.08% and 3.99% improvement over the vanilla-GRPO baselines.
CogCompTime: A Tool for Understanding Time in Natural Language (D18-2)

Copied to clipboard

Challenge: Existing systems that extract temporal information from text can be useful for natural language understanding.
Approach: They propose a system that extracts temporal information from text and normalizes it to a standard format.
Outcome: The proposed system achieves state-of-the-art performance and incorporates the most recent progress.
Learning to Decompose: Hypothetical Question Decomposition Based on Comparable Texts (2022.emnlp-main)

Copied to clipboard

Challenge: Existing approaches to end-to-end questionanswering assume that pre-trained language can decompose complex tasks into more straightforward sub-tasks.
Approach: They propose to use distant supervision to train decomposition-based transformers for large-scale parallel news.
Outcome: The proposed model improves on semantic parsing and on hotpotQA and strategyQA datasets by 20% to 30%.
XMC-Agent : Dynamic Navigation over Scalable Hierarchical Index for Incremental Extreme Multi-label Classification (2024.findings-acl)

Copied to clipboard

Challenge: Existing methods for XMC struggle with the growing set of labels due to their static label assumptions, and embedding-based methods struggle with complex mapping relationships due to late interaction paradigm.
Approach: They propose a large language model (LLM) powered agent framework for extreme multi-label classification, XMC-Agent, which can effectively learn, manage and predict the extremely large and dynamically increasing set of labels.
Outcome: The proposed framework can learn, manage and predict the extremely large and dynamically growing set of labels and achieves state-of-the-art performance on three standard datasets.
Generic Temporal Reasoning with Differential Analysis and Explanation (2023.acl-long)

Copied to clipboard

Challenge: Existing temporal reasoning models drop to random guessing on TODAY, suggesting that they heavily rely on spurious information rather than proper reasoning for temporal predictions.
Approach: They propose a task called TODAY that evaluates whether systems can correctly understand the effect of incremental changes in temporal relation distributions.
Outcome: The proposed task outperforms existing models, including GPT-3.5, on in-domain benchmarks while allowing for more appropriate annotations.
Self-Steering Optimization: Autonomous Preference Optimization for Large Language Models (2025.findings-acl)

Copied to clipboard

Challenge: Prior research focused on developing data generation methods, while insufficient attention has been paid to quality control mechanisms and often produces inaccurate and unhelpful data.
Approach: They propose an algorithm that automatically generates high-quality preference data, eliminating manual annotation requirements.
Outcome: The proposed algorithm outperforms baselines in human preference alignment and reward optimization.
Spatial Layouts in News Homepages Capture Human Preferences (2025.emnlp-main)

Copied to clipboard

Challenge: Homepage layouts are daily and manually curated by expert human news editors, and serve as a proxy for information prioritization.
Approach: They propose a pairwise preference model to capture ranked preferences between news items . they use a dataset of 3,000 news homepages captured twice daily over a five-year period .
Outcome: The proposed model infers editorial preference with over 0.7 F1 score (based on human trials) it also ranks local city council policies passed over ten years in San Francisco, evaluating their "newsworthiness".
ToW: Thoughts of Words Improve Reasoning in Large Language Models (2025.naacl-long)

Copied to clipboard

Challenge: Unlike other data augmentation methods, thoughts of words (TOW) views next-word prediction as a core reasoning task and injects fine-grained thoughts into pre-training texts.
Approach: They propose a training-time data-augmentation method called thoughts of words (TOW) that injects fine-grained thoughts directly into a next-word prediction task and teaches the model to understand how the observed next word is related to previous contexts.
Outcome: The proposed method reduces model hallucination by 10% and improves reasoning performance by 7% to 9% on average.
New Frontiers of Information Extraction (2022.naacl-tutorials)

Copied to clipboard

Challenge: Information extraction (IE) is the process of automatically extracting structural information from unstructured or semi-structured data.
Approach: This tutorial will provide an introduction to recent advances in IE by answering several important research questions.
Outcome: The tutorial will address several important research questions and outline directions for further investigation.
Evaluating Models’ Local Decision Boundaries via Contrast Sets (2020.findings-emnlp)

Copied to clipboard

Challenge: Standard test sets for supervised learning evaluate in-distribution generalization but are misleading when a dataset has systematic gaps.
Approach: They propose a more rigorous annotation paradigm for NLP that helps to close systematic gaps in the test data.
Outcome: The proposed model performs significantly lower on contrast sets than on the original test sets—up to 25% in some cases.
Expanding the Boundaries of Vision Prior Knowledge in Multi-modal Large Language Models (2026.eacl-long)

Copied to clipboard

Challenge: Existing research treats MLLMs as unified systems optimized through end-to-end training, but the impact of vision encoder’s prior knowledge is seldom investigated.
Approach: They propose a metric to quantify the effect of prior knowledge on MLLM performance by integrating prior knowledge at the vision encoder level into a training framework.
Outcome: The proposed training framework incorporates prior knowledge at the vision encoder level, and significantly boosts visual understanding capabilities of MLLMs.
Hidding the Ghostwriters: An Adversarial Evaluation of AI-Generated Student Essay Detection (2023.emnlp-main)

Copied to clipboard

Challenge: Large language models (LLMs) have exhibited remarkable capabilities in text generation tasks, but their utilization carries inherent risks, including plagiarism and the dissemination of fake news.
Approach: They propose to use a dataset to construct an AI-generated student essay that employs a range of text perturbation methods to evade detection.
Outcome: The proposed methods evade detection and maintain quality of the generated essays while avoiding plagiarism and fake news.
A Meta-framework for Spatiotemporal Quantity Extraction from Text (2022.acl-long)

Copied to clipboard

Challenge: a meta-framework for news events that extracts quantities from text is proposed . a previous work on news events focused on extracting event mentions, attributes, and relationships .
Approach: They propose a meta-framework for solving the NLP problem of spatiotemporal quantity extraction . they demonstrate the framework is general and extensible, and shareable crowdsourcing pipeline and baseline models are used .
Outcome: The proposed framework is general and extensible, the authors say . it can extract quantity from news streams, quickly respond to emergencies, investigate incidents .
Humanizing Machine-Generated Content: Evading AI-Text Detection through Adversarial Attack (2024.lrec-main)

Copied to clipboard

Challenge: Despite the development of large language models, there are still significant challenges in detecting whether text is generated by a machine.
Approach: They propose a framework for a broader class of adversarial attacks to perform minor perturbations in machine-generated content to evade detection.
Outcome: The proposed framework can be compromised in as little as 10 seconds, and improves over iterative adversarial learning.
End-to-End Chinese Speaker Identification (2022.naacl-main)

Copied to clipboard

Challenge: Existing methods for speaker identification in texts are incomplete and introduce errors that propagate and seriously affect the final output.
Approach: They propose to use speaker identification (SI) in texts to identify the speaker(s) for each utterance in texts.
Outcome: The proposed model can achieve comparable or better than previous state-of-the-art methods on all public SI datasets for Chinese.
PredictaBoard: Benchmarking LLM Score Predictability (2025.findings-acl)

Copied to clipboard

Challenge: Large Language Models (LLMs) fail unpredictably, demonstrating inconsistent success in even basic common sense reasoning tasks.
Approach: They propose a framework to evaluate the ability of score predictors to anticipate LLM errors on specific task instances from existing datasets.
Outcome: The proposed framework evaluates the ability of score predictors to anticipate LLM errors on specific task instances from existing datasets.
Towards Enhancing Health Coaching Dialogue in Low-Resource Settings (2022.coling-1)

Copied to clipboard

Challenge: Health coaching is cost-prohibitive due to its highly personalized nature.
Approach: They propose to build a health coaching dialogue system that converses with patients . they propose to use simplified NLU and NLG frameworks and mechanism-conditioned empathetic response generation.
Outcome: The proposed system generates more empathetic, fluent, and coherent responses . it outperforms the state-of-the-art in NLU tasks while requiring less annotations.
InfoGain-RAG: Boosting Retrieval-Augmented Generation through Document Information Gain-based Reranking and Filtering (2025.emnlp-main)

Copied to clipboard

Challenge: Retrieval-Augmented Generation (RAG) frameworks struggle with identifying whether retrieved documents meaningfully contribute to answer generation.
Approach: They propose a document-related metric to quantify the contribution of retrieved documents to correct answer generation.
Outcome: The proposed framework outperforms existing approaches on both single and multiple retrieval paradigms.
QA‐LIGN: Aligning LLMs through Constitutionally Decomposed QA (2025.findings-emnlp)

Copied to clipboard

Challenge: QA-LIGN decomposes monolithic rewards into interpretable principle-specific evaluations . scalar rewards obscure which objectives drive the training signal .
Approach: a new method decomposes monolithic rewards into interpretable principle-specific evaluations . QA-LIGN reduces attack success rates by up to 68.7% while maintaining a 0.67% false refusal rate .
Outcome: QA-LIGN reduces attack success rates by up to 68.7% while maintaining a 0.67% false refusal rate . the results outperform DPO and GRPO with state-of-the-art reward models given equivalent training .
Cross-lingual Entity Alignment with Incidental Supervision (2021.eacl-main)

Copied to clipboard

Challenge: Existing methods to match entities in multilingual knowledge graphs are insufficient, resulting in inconsistent seed alignment between KGs.
Approach: They propose a model that integrates multilingual KGs and monolingual text corpora in a shared embedding scheme and a self-learning based alignment learning process to induce correspondence between entities and lexemes.
Outcome: The proposed model significantly outperforms state-of-the-art methods on benchmark datasets and significantly outpersts existing methods.
CogCompNLP: Your Swiss Army Knife for NLP (L18-1)

Copied to clipboard

Challenge: a corpus-reader module supports popular corpora, feature extraction and annotation modules for semantic and syntactic tasks.
Approach: They propose a library that provides modules to address different challenges . they provide a corpus-reader module that supports popular corpora in the NLP community .
Outcome: The proposed library simplifies the process of design and development of NLP applications by providing modules to address different challenges.
Indirectly Supervised Natural Language Processing (2023.acl-tutorials)

Copied to clipboard

Challenge: a tutorial on indirect supervision addresses challenges in ML for NLP . conventional approaches to NLP use taskspecific labeled examples of a large volume . indirect supervision is useful for a wide range of NLP tasks, but it is not enough for decoders .
Approach: This tutorial aims to address questions about indirect supervision in machine learning . authors discuss indirect supervision from T′ that handles T with outputs spanning from a moderate size to an open space .
Outcome: This tutorial aims to answer questions about how to provide supervision for ML tasks . it will discuss indirect supervision from T′ that handles T with outputs spanning from a moderate size to an open space .
Zero-Shot Open Entity Typing as Type-Compatible Grounding (D18-1)

Copied to clipboard

Challenge: Existing approaches to entity typing have limited flexibility to transfer across text genres and generalize to new type taxonomies.
Approach: They propose a zero-shot entity typing approach that requires no annotated data and can flexibly identify newly defined types.
Outcome: The proposed system outperforms state-of-the-art supervised NER systems on a broad range of datasets and on 'biological domain' it is competitive with supervised systems and outperformed on out-of training datasets.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations