Papers by Qi Zeng

40 papers
Verified Critical Step Optimization for LLM Agents (2026.findings-acl)

Copied to clipboard

Challenge: Critical Step Optimization (CSO) focuses preference learning on verified critical steps where alternative actions demonstrably flip task outcomes from failure to success.
Approach: They propose a method which focuses preference learning on verified critical steps where alternative actions demonstrably flip task outcomes from failure to success.
Outcome: The proposed method outperforms the existing methods on GAIA-Text-103 and XBench-DeepSearch while requiring supervision at only 16% of trajectory steps.
Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitution (2021.emnlp-main)

Copied to clipboard

Challenge: Existing methods to defend against adversarial word-substitution attacks have not been evaluated or compared in a systematic manner.
Approach: They propose to compare different defense methods under representative adversarial attacks . they propose a method that improves the robustness of neural text classifiers against such attacks a .
Outcome: The proposed method improves robustness of neural text classifiers against such attacks by a significant margin.
Cross-media Structured Common Space for Multimedia Event Extraction (2020.acl-main)

Copied to clipboard

Challenge: We propose a new task to extract events and their arguments from multimedia documents . traditional methods target text, images or videos, but multimedia content is distributed via multimedia .
Approach: They propose a method that encodes structured representations of semantic information from textual and visual data into a common embedding space.
Outcome: The proposed method achieves 4.0% and 9.8% absolute gains on text event argument role labeling and visual event extraction.
Exploring the Capability of Multimodal LLMs with Yonkoma Manga: The YManga Dataset and Its Challenging Tasks (2024.findings-emnlp)

Copied to clipboard

Challenge: YManga dataset is the first specifically designed for yonkoma manga understanding .
Approach: They propose to use a dataset of 1,015 yonkoma strips with 10,150 human annotations to define three tasks for panel sequence detection, intent generation and description generation for masked panels.
Outcome: The proposed dataset contains 1,015 high-quality yonkoma strips with 10,150 human annotations.
Sarcasm-R1: Enhancing Sarcasm Detection through Focused Reasoning (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing methods for sarcasm detection are limited by supervised learning or prompt engineering . a new approach decomposes sarcasm detection into three dimensions: language, context, and emotion .
Approach: They propose a method that decomposes sarcasm detection into three dimensions: language, context, and emotion.
Outcome: The proposed method outperforms state-of-the-art methods in most cases.
CRITICTOOL: Evaluating Self-Critique Capabilities of Large Language Models in Tool-Calling Error Scenarios (2025.emnlp-main)

Copied to clipboard

Challenge: a number of tools are used to perform complex tasks, but the tool utilization process can cause errors.
Approach: They propose a critique evaluation benchmark for tool learning that analyzes function-calling errors on tool evaluation benchmarks.
Outcome: The proposed critique evaluation benchmark holds diverse tool-use errors with varying complexities, which better reflects real-world scenarios.
ReviewRL: Towards Automated Scientific Review with RL (2025.emnlp-main)

Copied to clipboard

Challenge: Existing automated review systems struggle with factual accuracy, rating consistency, and analytical depth.
Approach: They propose a framework for generating comprehensive and factually grounded scientific paper reviews using supervised fine-tuning and reinforcement learning.
Outcome: The proposed framework outperforms existing methods on ICLR 2025 papers.
A Skeleton-Based Model for Promoting Coherence Among Sentences in Narrative Story Generation (D18-1)

Copied to clipboard

Challenge: Existing models for narrative story generation lack semantic dependency among sentences.
Approach: They propose a skeleton-based model that generates the most critical phrases and expands them to a complete sentence.
Outcome: The proposed model can generate significantly more coherent stories according to human evaluation and automatic evaluation.
The Confidence Dichotomy: Analyzing and Mitigating Miscalibration in Tool-Use Agents (2026.acl-long)

Copied to clipboard

Challenge: a fundamental pillar of trustworthiness is calibration, which refers to an agent’s ability to express confidence that reliably reflects its actual performance.
Approach: They propose a reinforcement learning framework that jointly optimizes task accuracy and calibration, supported by a holistic benchmark of reward designs.
Outcome: The proposed framework improves calibration across tool types and shows that trained agents achieve superior calibration and exhibit robust generalization from local training environments to noisy web settings and to distinct domains such as mathematical reasoning.
Code-Switching Information Retrieval: Benchmarks, Analysis, and the Limits of Current Retrievers (2026.findings-acl)

Copied to clipboard

Challenge: a new study examines the performance of code-switching IR in monolingual contexts . code-witching is a pervasive linguistic phenomenon in global communication .
Approach: They propose a benchmark to evaluate code-switching IR in monolingual contexts . they propose CS-MTEB, which measures performance declines of up to 27% .
Outcome: The proposed benchmark shows that code-switching performance is degraded by 27% . the proposed benchmark is based on a dataset of mixed-language queries .
A Web Scale Entity Extraction System (2021.findings-emnlp)

Copied to clipboard

Challenge: Existing systems for large-scale entity extraction are limited by the scale and variety of data available on internet platforms.
Approach: They propose to build an entity extraction system for multiple document types at large scale using multi-modal Transformers.
Outcome: The proposed system extracts multiple types of entities from multiple document types at large scale using multi-modal Transformers.
Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction (2023.emnlp-main)

Copied to clipboard

Challenge: Existing evaluation benchmarks focus on pairwise matching, ignoring robustness . current models exhibit frustrating degradation, with a maximum drop of 23.43 F1 score .
Approach: They propose a benchmark that simulates the evaluation of open information extraction models in the real world . they perform experiments on typical models published in the last decade and a representative large language model .
Outcome: The proposed model is rated robust on a knowledge-invariant clique with different syntactic and expressive forms.
Context Reasoner: Incentivizing Reasoning Capability for Contextualized Privacy and Safety Compliance via Reinforcement Learning (2025.emnlp-main)

Copied to clipboard

Challenge: Current mitigation strategies fail to preserve contextual reasoning capabilities in risky scenarios, leading to systemic risks for legal compliance.
Approach: They propose to use reinforcement learning with a rule-based reward to incentivize contextual reasoning capabilities while enhancing compliance with safety and privacy norms.
Outcome: The proposed model outperforms Qwen2.5-7B-Instruct model in safety and privacy benchmarks and achieves +8.58% accuracy improvement.
Asking Clarification Questions in Knowledge-Based Question Answering (D19-1)

Copied to clipboard

Challenge: Existing clarification datasets with limited annotated examples do not address ambiguous phenomena.
Approach: They propose a dataset that allows users to ask clarification questions using open-domain examples.
Outcome: The proposed model achieves better performance than strong baselines and provides new challenges.
Automatic Prompt Engineering for Scalable Prompt Inversion in Text-to-Image Ad Generation (2026.acl-industry)

Copied to clipboard

Challenge: PRISM-DUEL is a black-box framework that formalizes prompt optimization as Automatic Prompt Engineering (APE) PRIMS-DUEl is motivated by advertising workflows requiring low-latency, diverse variants faithful to a human-designed ad.
Approach: They propose a black-box framework that formalizes prompt optimization as Automatic Prompt Engineering (APE) they obtain label-free pairwise preferences and rationales from an LLM judge over pairs of generated images and use a dueling-bandit optimizer to optimize a prompt for generating controlled variations while matching the reference ad's visual content.
Outcome: The proposed framework preserves visual similarity and semantic faithfulness while increasing diversity.
An Auto-Encoder Matching Model for Learning Utterance-Level Semantic Dependency in Dialogue Generation (D18-1)

Copied to clipboard

Challenge: Experimental results show that our model can generate semantically coherent responses compared to baseline models.
Approach: They propose an Auto-Encoder Matching model to learn utterance-level semantic dependency . their model contains two auto-encoders and one mapping module .
Outcome: Experimental results show that the proposed model can generate high coherence and fluency compared to baseline models.
OctoBench: Benchmarking Scaffold-Aware Instruction Following in Repository-Grounded Agentic Coding (2026.acl-long)

Copied to clipboard

Challenge: coding scaffolds that follow heterogeneous instructions remain under-examined in software engineering . coding models are capable software agents, but their ability to follow constraints remains under-explored .
Approach: They introduce OctoBench, which benchmarks scaffold-aware instruction following in agentic coding.
Outcome: The proposed benchmark aims to accelerate the development of more scaffold-aware agents.
Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models (2025.findings-acl)

Copied to clipboard

Challenge: Chain-of-Thought (CoT) reasoning has improved the performance of large language models (LLMs) however, the detailed reasoning process in CoT often incurs long generation times and high computational costs due to the inclusion of unnecessary steps.
Approach: They propose a method to identify critical reasoning steps using perplexity as a measure of their importance.
Outcome: The proposed method achieves a better balance between reasoning accuracy and efficiency of CoT.
Harnessing Consistency for Robust Test-Time LLM Ensemble (2026.findings-eacl)

Copied to clipboard

Challenge: Existing efforts to improve LLM ensemble quality have focused on model consistency, but failures are often due to heterogeneous tokenization schemes and varying model expertise.
Approach: They propose a plug-and-play technique that harnesses model consistency for robust LLM ensemble.
Outcome: The proposed technique improves ensemble performance and robustness against erroneous signals.
Improving Consistency for Text Summarization with Energy Functions (2023.findings-emnlp)

Copied to clipboard

Challenge: Current abstractive summarization models generate inconsistent content due to the inherently noisy dataset and the discrepancy between maximum likelihood estimation based training objectives and consistency measurements.
Approach: They propose a new consistency taxonomy that categorizes inconsistent content into faithfulness, factuality, and self-supportiveness.
Outcome: Experiments on XSUM and CNN/DM datasets show that EnergySum mitigates the trade-off between accuracy and consistency.
Reinforcement Learning on Pre-Training Data (2026.acl-long)

Copied to clipboard

Challenge: Recent progress in large language models is driven by scaling of training compute through pre-training with nexttoken prediction (NTP) or post-training (RL) Pre-training using NTP enables models to acquire extensive knowledge and skills from general data, but it suffers from data inefficiency and catastrophic forgetting in continual learning settings.
Approach: They propose to scale training compute through pre-training with next-token prediction (NTP) or post-training by scaling reinforcement learning (RL) to improve learning from general data.
Outcome: Experiments on multiple benchmarks and models show that the proposed approach improves continual pre-training and provides a strong foundation for post-training on Qwen3-8B-Base.
A General Framework to Enhance Fine-tuning-based LLM Unlearning (2025.findings-acl)

Copied to clipboard

Challenge: Existing approaches to remove copyrighted and privacy-sensitive data from Large Language Models (LLMs) have been proposed to remove specific data from LLMs without requiring full retraining.
Approach: They propose a general framework that enhances the utility of fine-tuning-based methods by distinguishing target data and suppressing related generations.
Outcome: The proposed framework improves the unlearning and utility of fine-tuning-based methods by distinguishing the target data and suppressing related generations.
Review-Driven Multi-Label Music Style Classification by Exploiting Style Correlations (N19-1)

Copied to clipboard

Challenge: Several methods have been proposed for automatic music style classification, but they are limited in two aspects.
Approach: They propose a deep learning approach to automatically learn and exploit style correlations by reviewing music reviews on websites.
Outcome: The proposed approach performs well in capturing style correlations.
OpenAttack: An Open-source Textual Adversarial Attack Toolkit (2021.acl-demo)

Copied to clipboard

Challenge: Various attack models are distinct and implemented with different programming frameworks and settings, which hinders quick utilization and fair comparison of attack models.
Approach: They propose an open-source textual adversarial attack toolkit to solve these issues by combining 15 typical attack models into one toolkit.
Outcome: The proposed toolkit supports all attack types, multilinguality, and parallel processing.
Connecting the Dots: Event Graph Schema Induction with Path Language Modeling (2020.emnlp-main)

Copied to clipboard

Challenge: Existing methods to automate event extraction focus on uncertainty, re-occurring events and multiple hypotheses.
Approach: They propose a new Event Graph Schema where two event types are connected through multiple paths involving entities that fill important roles in a coherent story.
Outcome: The proposed model is highly effective at inducing salient and coherent schemas.
Seeing is Believing, but How Much? A Comprehensive Analysis of Verbalized Calibration in Vision-Language Models (2025.emnlp-main)

Copied to clipboard

Challenge: Existing approaches to quantify uncertainty are limited in vision-language models . however, current models display notable miscalibration across diverse tasks and settings .
Approach: They evaluate verbalized confidence in vision-language models using visual reasoning . they propose a prompting strategy that improves confidence alignment in multimodal settings .
Outcome: The proposed method improves confidence alignment across multimodal settings.
ChartEdit: How Far Are MLLMs From Automating Chart Analysis? Evaluating MLLMs’ Capability via Chart Editing (2025.findings-acl)

Copied to clipboard

Challenge: Existing evaluations of multimodal large language models rely on limited case studies . however, they lack the ability to generate accurate edits according to the instructions .
Approach: They propose a benchmark for chart editing that includes 1,405 edit instructions applied to 233 real-world charts.
Outcome: The proposed benchmark includes 1,405 diverse editing instructions applied to 233 real-world charts.
LexicalAT: Lexical-Based Adversarial Reinforcement Training for Robust Sentiment Classification (D19-1)

Copied to clipboard

Challenge: Existing text classification models are fragile and sensitive to simple perturbations.
Approach: They propose a generator-classifier adversarial training approach to improve classification models . they use a large-scale lexical knowledge base to generate attacking examples .
Outcome: The proposed approach outperforms strong baselines and reduces test errors on neural networks.
For-Value: Efficient Forward-Only Data Valuation for finetuning LLMs and VLMs (2026.acl-long)

Copied to clipboard

Challenge: Existing methods for data valuation rely on gradient computations, making them prohibitive for billion-parameter models.
Approach: They propose a forward-only data valuation framework that enables efficient batch-scalable value estimation while maintaining effectiveness.
Outcome: The proposed framework matches or outperforms gradient-based baselines in detecting influential data and mislabeled data while achieving significant efficiency improvements.
Enhancing Large Vision-Language Models with Ultra-Detailed Image Caption Generation (2025.emnlp-main)

Copied to clipboard

Challenge: Existing pipelines for generating high-quality, ultra-detailed image captions are limited by the scarcity of image caption data.
Approach: They propose a pipeline for generating high-quality, ultra-detailed image captions that integrates both pre-processing and post-processor stages.
Outcome: The proposed pipeline improves LVLMs' perception and cognitive abilities across multiple vision-language benchmarks.
Interpretable Automatic Fine-grained Inconsistency Detection in Text Summarization (2023.findings-acl)

Copied to clipboard

Challenge: Existing factual consistency evaluation approaches for text summarization provide binary predictions and limited insights into the weakness of summarizing systems.
Approach: They propose a task of fine-grained inconsistency detection to detect inconsistent summaries . they represent facts with semantic frames extracted by semantic role labeling and highlight related semantic frames to predict inconsistance .
Outcome: The proposed model outperforms baseline models and provides evidence to support or refute the summary.
MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation (2025.emnlp-main)

Copied to clipboard

Challenge: Existing large language model evaluation benchmarks focus on English, while current multilingual tasks lack parallel questions that specifically assess cross-lingual reasoning abilities.
Approach: They propose a comprehensive benchmark covering 29 languages, built on an English benchmark.
Outcome: The MMLU-ProX is a comprehensive benchmark covering 29 languages, built on an English benchmark.
Fin-STAR: Structure-as-Semantics to Resolve Implicitness in Financial Retrieval (2026.findings-acl)

Copied to clipboard

Challenge: Existing Retrieval-Augmented Generation systems treat structure as a physical navigational skeleton rather than intrinsic semantic knowledge.
Approach: They propose a framework that redefining hierarchy as intrinsic semantics and uses snippets to enrich hierarchical lineage.
Outcome: The proposed framework outperforms state-of-the-art hierarchical and graph-based benchmarks on FinTierQA Gold.
Sketching as a Tool for Understanding and Accelerating Self-attention for Long Sequences (2022.naacl-main)

Copied to clipboard

Challenge: Existing models for long sequences are not efficient due to the quadratic space and time complexity of the self-attention modules.
Approach: They propose to reduce the quadratic complexity to linear (modulo logarithmic factors) by low-dimensional projection and row selection.
Outcome: The proposed methods outperform transformer-based models with smaller time/space footprint on the Long Range Arena benchmark.
VisFinEval: A Scenario-Driven Chinese Multimodal Benchmark for Holistic Financial Understanding (2025.emnlp-main)

Copied to clipboard

Challenge: Existing benchmarks focus on text comprehension, but MLLMs lack the ability to integrate visual data over financial visuals.
Approach: They evaluate 21 state-of-the-art multimodal large language models in a zero-shot setting . they use an annotated question–answer pair from eight common financial image modalities .
Outcome: The new benchmark outperforms existing models but trailed financial experts by 14 percentage points.
Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach (P18-1)

Copied to clipboard

Challenge: Existing studies for sentiment-to-sentiment "translation" only change the underlying sentiment and fail to keep the semantic content.
Approach: They propose a cycled reinforcement learning method that combines neutralization module and emotionalization module.
Outcome: The proposed method outperforms state-of-the-art systems on Yelp and Amazon review datasets.
Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios (2024.findings-acl)

Copied to clipboard

Challenge: Existing benchmarks focus on simple synthesized queries that do not reflect real-world complexity, thereby offering limited perspectives in evaluating tool utilization.
Approach: They propose a benchmark to evaluate LLMs’ ability in tool utilization within real-world scenarios.
Outcome: The proposed benchmark improves LLMs’ ability in tool utilization within real-world scenarios and eliminates the restriction of pre-defined toolset.
EA2E: Improving Consistency with Event Awareness for Document-Level Argument Extraction (2022.findings-naacl)

Copied to clipboard

Challenge: Recent work on document-level event argument extraction models each individual event in isolation and therefore causes inconsistency among extracted arguments across events.
Approach: They propose an event-aware argument extraction model with augmented context to improve consistency . they hypothesize that participants tend to play consistent roles across multiple events in a document .
Outcome: The proposed model improves consistency and accuracy of arguments extracted from documents.
Alleviating Performance Degradation Caused by Out-of-Distribution Issues in Embedding-Based Retrieval (2025.findings-emnlp)

Copied to clipboard

Challenge: Recent studies reveal query out-of-distribution issues degrading ANN performance . a distribution regularizer is introduced into the encoder training objective to encourage alignment between query and base embeddings.
Approach: They introduce a distribution regularizer into the encoder training objective to encourage alignment between query and base embeddings.
Outcome: The proposed method consistently improves retrieval performance across multiple datasets.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations