Papers by Yi Zeng

Soft Language Clustering for Multilingual Model Pre-training (2023.acl-long)

Challenge:	Out-of-distribution (OOD) detection is essential for reliable and trustworthy machine learning.
Approach:	They propose to apply world knowledge to enhance OOD detection performance through selective generation from large language models (LLMs) they propose to extract visual objects from each image to fully capitalize on the aforementioned world knowledge.
Outcome:	The proposed method outperforms the state-of-the-art on visual OOD detection on in-distribution (ID) samples.

Copied to clipboard

Jiali Zeng, Yufan Jiang, Yongjing Yin, Yi Jing, Fandong Meng, Binghuai Lin, Yunbo Cao, Jie Zhou

Multilinguality

Challenge:	Multilingual pre-trained language models have demonstrated impressive (zero-shot) cross-lingual transfer abilities, however, their performance is hindered when the target language has distant typology from the source language or when pre-training data is limited in size.
Approach:	They propose a method that contextually retrieves prompts as flexible guidance for encoding instances conditionally.
Outcome:	The proposed method improves on the XTREME task and also for low-resource languages in unsupervised sentence retrieval.

A Skeleton-Based Model for Promoting Coherence Among Sentences in Narrative Story Generation (D18-1)

Copied to clipboard

Jingjing Xu, Xuancheng Ren, Yi Zhang, Qi Zeng, Xiaoyan Cai, Xu Sun

Text Generation

Challenge:	Existing models for narrative story generation lack semantic dependency among sentences.
Approach:	They propose a skeleton-based model that generates the most critical phrases and expands them to a complete sentence.
Outcome:	The proposed model can generate significantly more coherent stories according to human evaluation and automatic evaluation.

DynaMaR: Dynamic Prompt with Mask Token Representation (2022.emnlp-industry)

Copied to clipboard

Xiaodi Sun, Sunny Rajagopalan, Priyanka Nigam, Weiyi Lu, Yi Xu, Iman Keivanloo, Belinda Zeng, Trishul Chilimbi

CogToM: A Comprehensive Theory of Mind Benchmark inspired by Human Cognition for Large Language Models (2026.acl-long)

Challenge:	Recent research shows that large language models pretrained using unsupervised approaches can achieve significant performance improvement on many downstream tasks.
Approach:	They propose an unsupervised approach to fine-tuning large language models using unsupervised approaches to many downstream tasks.
Outcome:	The proposed approach improves on four e-commerce applications and can achieve an average improvement of 10% in few-shot settings and 3.7% in data-rich settings over the standard approach.

Copied to clipboard

Haibo Tong, Zeyang Yue, Feifei Zhao, Erliang Lin, Lu Jia, Ruolin Chen, Yinqian Sun, Qian Zhang, Yi Zeng

Linguistics & Cognitive NLP

Challenge:	Existing benchmarks for Large Language Models (LLMs) are limited to false belief tasks, highlighting bottlenecks in specific dimensions.
Approach:	They propose a benchmark to evaluate Large Language Models' Theory of Mind capabilities . they evaluate 8000 bilingual instances across 46 paradigms and validated by 49 human annotators .
Outcome:	The proposed benchmark reveals performance heterogeneities and bottlenecks in 22 representative models.

HelpSteer3: Human-Annotated Feedback and Edit Data to Empower Inference-Time Scaling in Open-Ended General-Domain Tasks (2025.acl-long)

Copied to clipboard

Zhilin Wang, Jiaqi Zeng, Olivier Delalleau, Daniel Egert, Ellie Evans, Hoo-Chang Shin, Felipe Soares, Yi Dong, Oleksii Kuchaiev

Challenge:	Inference-Time Scaling is critical to the success of recent models such as OpenAI o1 and DeepSeek R1 . however, many techniques require tasks to have answers that can be verified .
Approach:	They use data to train dedicated Feedback and Edit Models capable of inference-time scaling for open-ended tasks.
Outcome:	The proposed model can reach SoTA performance on Arena Hard at 92.7 as of 5 Mar 2025.

RTADev: Intention Aligned Multi-Agent Framework for Software Development (2025.findings-acl)

Copied to clipboard

Jie Liu, Guohua Wang, Ronghui Yang, Jiajie Zeng, Mengchen Zhao, Yi Cai

Evaluating and Enhancing the Robustness of Neural Network-based Dependency Parsing Models with Adversarial Examples (2020.acl-main)

Challenge:	Large Language Models (LLMs) are efficient assistants to humans in software development tasks, but they can cause errors during the development process.
Approach:	They propose an intention aligned multi-agent framework that ensures that all agents work based on a consensus.
Outcome:	The proposed framework reduces errors and improves the quality of generated software code.

Copied to clipboard

Xiaoqing Zheng, Jiehang Zeng, Yi Zhou, Cho-Jui Hsieh, Minhao Cheng, Xuanjing Huang

Robustness in NLP

Challenge:	Previously studies focused on semantic tasks such as sentiment analysis, question answering and reading comprehension.
Approach:	They propose two approaches to study where and how adversarial examples exist in dependency parsing . they use a state-of-the-art parser to find adversarials in existing texts .
Outcome:	The proposed approaches show that adversarial examples exist in dependency parsing . they show that up to 77% of input examples admit adversarials .

PSSAT: A Perturbed Semantic Structure Awareness Transferring Method for Perturbation-Robust Slot Filling (2022.coling-1)

Copied to clipboard

Guanting Dong, Daichi Guo, Liwen Wang, Xuefeng Li, Zechen Wang, Chen Zeng, Keqing He, Jinzheng Zhao, Hao Lei, Xinyue Cui, Yi Huang, Junlan Feng, Weiran Xu

Robustness in NLP

Challenge:	Existing slot filling models memorize inherent patterns of entities and contexts from training data.
Approach:	They propose a perturbed semantic structure awareness transferring method for slot filling models . they use two MLM-based training strategies to learn contextual semantic structure and word distribution .
Outcome:	The proposed method outperforms existing methods and gains strong generalization while preventing model from memorizing inherent patterns of entities and contexts.

Reinforcement Learning on Pre-Training Data (2026.acl-long)

Copied to clipboard

Siheng Li, Kejiao Li, Zenan Xu, Guanhua Huang, Kun Li, Haoyuan Wu, null Wujiajia, Zihao Zheng, Chenchen Zhang, Kun Shi, Xue Gong, Qi Yi, Ruibin Xiong, Tingqiang Xu, Yuhao Jiang, Jianfeng Yan, Yuyuan Zeng, Guanghui Xu, Jinbao Xue, Zhijiang xu, Zheng Fang, Shuai LI, Qibin Liu, Xiaoxue Li, Zhuoyu Li, Yangyu Tao, Fei Gao, Cheng Jiang, Bochao Wang, Kai Liu, Jianchen Zhu, Wai Lam, Bo Zhou, Di Wang

Language Models

Challenge:	Recent progress in large language models is driven by scaling of training compute through pre-training with nexttoken prediction (NTP) or post-training (RL) Pre-training using NTP enables models to acquire extensive knowledge and skills from general data, but it suffers from data inefficiency and catastrophic forgetting in continual learning settings.
Approach:	They propose to scale training compute through pre-training with next-token prediction (NTP) or post-training by scaling reinforcement learning (RL) to improve learning from general data.
Outcome:	Experiments on multiple benchmarks and models show that the proposed approach improves continual pre-training and provides a strong foundation for post-training on Qwen3-8B-Base.

The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG) (2024.findings-acl)

Copied to clipboard

Shenglai Zeng, Jiankun Zhang, Pengfei He, Yiding Liu, Yue Xing, Han Xu, Jie Ren, Yi Chang, Shuaiqiang Wang, Dawei Yin, Jiliang Tang

Graph-Reward-SQL: Execution-Free Reinforcement Learning for Text-to-SQL via Graph Matching and Stepwise Reward (2025.findings-emnlp)

Challenge:	Retrieval-augmented generation (RAG) is a powerful technique to facilitate language model generation with proprietary and private data, where data privacy is . a privacy issue that is currently under-explored, is posed by RAG.
Approach:	They propose to use retrieval-augmented generation (RAG) to facilitate language model generation with proprietary and private data where data privacy is a pivotal concern.
Outcome:	The proposed attack methods demonstrate that RAG can mitigate the old risks, i.e., leakage of the LLMs’ training data.

Copied to clipboard

Han Weng, Puzhen Wu, Cui Longjie, Yi Zhan, Boyi Liu, Yuanfeng Song, Dun Zeng, Yingxiang Yang, Qianru Zhang, Dong Huang, Xiaoming Yin, Yang Sun, Xing Chen

ODE Transformer: An Ordinary Differential Equation-Inspired Model for Sequence Generation (2022.acl-long)

Challenge:	Existing methods to enhance performance of large language models (LLMs) on Text-to-SQL tasks rely on execution-based or LLM-based reward models.
Approach:	They propose a reward model framework for RL-based Text-to-SQL that employs the GMNScore outcome reward model.
Outcome:	The proposed reward model outperforms existing reward models on standard benchmarks including Spider and BIRD.

Copied to clipboard

Bei Li, Quan Du, Tao Zhou, Yi Jing, Shuhan Zhou, Xin Zeng, Tong Xiao, JingBo Zhu, Xuebo Liu, Min Zhang

Language Models

Challenge:	Residual networks are an Euler discretization of solutions to Ordinary Differential Equations (ODE).
Approach:	They propose a residual block of layers in Transformer that can be described as a higher-order solution to ODE.
Outcome:	The proposed architecture can gain large improvements over strong baselines at a slight cost in inference efficiency.

Constructing Procedural Graphs with Multiple Dependency Relations: A New Dataset and Baseline (2023.findings-acl)

Copied to clipboard

Haopeng Ren, Yushi Zeng, Yi Cai, Bihan Zhou, Zetao Lian

RU22Fact: Optimizing Evidence for Multilingual Explainable Fact-Checking on Russia-Ukraine Conflict (2024.lrec-main)

Challenge:	Existing methods to structure procedural knowledge focus on representing descriptive knowledge but ignore another commonsense knowledge-Procedural Knowledge.
Approach:	They propose to generate flow graphs from procedural documents by extracting sequential dependency between sentences and missing two important dependencies in procedural document.
Outcome:	The proposed method can generate flow graphs from unstructured documents with syntactic information and discourse structures.

Copied to clipboard

Yirong Zeng, Xiao Ding, Yi Zhao, Xiangyu Li, Jie Zhang, Chao Yao, Ting Liu, Bing Qin

FaStFact: Faster, Stronger Long-Form Factuality Evaluations in LLMs (2025.findings-emnlp)

Challenge:	Existing methods to verify factuality of claims do not provide sufficient evidence for explainable fact-checking systems.
Approach:	They propose a method to automatically retrieve and summarize evidence from the Web and a novel multilingual explainable fact-checking dataset on the Russia-Ukraine conflict in 2022.
Outcome:	The proposed method can retrieve and summarize evidence from the Web and generate explanations in 16 languages.

Copied to clipboard

Yingjia Wan, Haochen Tan, Xiao Zhu, Xinyu Zhou, Zhiwei Li, Qingsong Lv, Changxuan Sun, Jiaqi Zeng, Yi Xu, Jianqiao Lu, Yinhong Liu, Zhijiang Guo

EDU-CIRCUIT-HW: Evaluating Multimodal Large Language Models on Real-World University-Level STEM Student Handwritten Solutions (2026.findings-acl)

Challenge:	Prior evaluation pipelines fail to evaluate factuality of long-form LLMs due to inefficiency and costly human assessment.
Approach:	They propose a fast and strong evaluation pipeline that can evaluate factuality of long-form LLMs . they propose 'faStFact' to reduce cost of web searching and inference calling .
Outcome:	The proposed evaluation pipeline achieves highest alignment with human evaluation and efficiency among existing baselines.

Copied to clipboard

Weiyu Sun, Liangliang Chen, Yongnuo Cai, Huiru Xie, Yi Zeng, Ying Zhang

Language Models

Challenge:	Multimodal Large Language Models (MLLMs) are a promising tool for traditional education but lack authentic and domain-specific benchmarks to accurately interpret student handwritten solutions.
Approach:	They propose to use MLLMs to interpret unconstrained STEM student handwritten solutions with intertwined mathematical formulas, diagrams, and textual reasoning to bridge this gap.
Outcome:	The proposed model can detect and rectify recognition errors with minimal human intervention on unseen student solutions.

Grounded Multimodal Procedural Entity Recognition for Procedural Documents: A New Dataset and Baseline (2024.lrec-main)

Copied to clipboard

Haopeng Ren, Yushi Zeng, Yi Cai, Zhenqi Ye, Li Yuan, Pinli Zhu

Information Extraction & Text Mining

Challenge:	Existing methods to extract procedural knowledge from documents focus on text-only settings, which is insufficient for entity disambiguation.
Approach:	They propose a model to detect the entity and the corresponding bounding box groundings in images.
Outcome:	The proposed model detects the entity and the corresponding bounding box groundings in image (i.e., visual entities) it is based on a dataset of a WikiHow 1 and EHow 2 document and the results are compared with existing models.

Beyond Text: Unveiling Privacy Vulnerabilities in Multi-modal Retrieval-Augmented Generation (2025.emnlp-main)

Copied to clipboard

Jiankun Zhang, Shenglai Zeng, Jie Ren, Tianqi Zheng, Hui Liu, Xianfeng Tang, Hui Liu, Yi Chang

CoGR-MoE: Concept-Guided Expert Routing with Consistent Selection and Flexible Reasoning for Visual Question Answering (2026.findings-acl)

Challenge:	Large Multimodal Models (LMMs) can process text, images, and audio, but they introduce privacy vulnerabilities.
Approach:	They propose a compositional structured prompt attack to exploit MRAG privacy vulnerabilities . they show that LMMs can generate outputs resembling retrieved content .
Outcome:	The proposed approach generates outputs resembling retrieved content and exposes sensitive information.

Copied to clipboard

Xiyin Zeng, Yi Lu, Hao Wang

RISER: Orchestrating Latent Reasoning Skills for Adaptive Activation Steering (2026.findings-acl)

Challenge:	Recent work has begun to address routing instability in VQA models by grouping similar concepts or routing based on examples.
Approach:	They propose a Concept-Guided Routing framework which incorporates semantics of the answer options to guide expert selection in the training phase.
Outcome:	The proposed framework delivers strong performance across multiple VQA tasks, demonstrating the effectiveness of the proposed framework.

Copied to clipboard

Wencheng Ye, Xiaoyang Yuan, Yi Bin, Hengyu Jin, Liang Peng, Pengpeng Zeng, Heng Tao Shen

Fin-STAR: Structure-as-Semantics to Resolve Implicitness in Financial Retrieval (2026.findings-acl)

Challenge:	Existing methods for domain-specific reasoning with large language models require updating parameter updates.
Approach:	They propose a plug-and-play intervention framework that adaptively steers LLM reasoning in activation space.
Outcome:	The proposed framework achieves zero-shot accuracy improvements of 3.4–6.5% over the base model while outperforming chain-of-thought-style reasoning with 2–3 higher token efficiency and robust accuracy gains.

Copied to clipboard

Yu Zou, Yan Chen, Lida He, Qi Zhou, Xiaorui Zhou, Aixi Zhong, Yi Wang, Wei Li, Qingyu Wang, Jiatao Li, Wei Gong, Jialei Zeng, Jingmei Zhao, Ke Jiang, Qing Li

Information Retrieval

Challenge:	Existing Retrieval-Augmented Generation systems treat structure as a physical navigational skeleton rather than intrinsic semantic knowledge.
Approach:	They propose a framework that redefining hierarchy as intrinsic semantics and uses snippets to enrich hierarchical lineage.
Outcome:	The proposed framework outperforms state-of-the-art hierarchical and graph-based benchmarks on FinTierQA Gold.

SeDev: Structured Semantic Exploration for LLM-Driven Code Generation (2026.acl-long)

Copied to clipboard

Ronghui Yang, Jie Liu, Jiajie Zeng, Jiexin Wang, Jiuchuan Jiang, Bo An, Yi Cai, Mengchen Zhao

Programming Languages in NLP

Challenge:	Large Language Models (LLMs) have shown remarkable capabilities in automating code generation, but they suffer from insufficient exploration of the vast solution space.
Approach:	They propose a large-scale LLM-driven code generation framework that efficiently finds high-quality solutions in only a few iterations.
Outcome:	The proposed framework outperforms baselines while maintaining reasonable time and computational costs.

UniCorn: Towards Self-Improving Unified Multimodal Models through Self-Generated Supervision (2026.acl-long)

Copied to clipboard

Zhen Fang, Ruiyan Han, XinYu Sun, Yuchen Ma, Ziheng Wang, Yu Zeng, Zehui Chen, Lin Chen, Wenxuan Huang, Wei-Jie Xu, Yi Cao, Feng Zhao

Attention Basin: Why Contextual Position Matters in Large Language Models (2026.acl-long)

Challenge:	Unified Multimodal Models have achieved remarkable success in cross-modal comprehension, but a gap persists in their ability to translate internal knowledge into faithful and controllable synthesis.
Approach:	They propose a self-improvement framework that partitions a single UMM into three collaborative roles: Proposer, Solver, and Judge.
Outcome:	The proposed framework improves on TIIF, DPG, CompBench and UniCycle benchmarks.

Copied to clipboard

Zihao Yi, Zhenqing Ling, Delong Zeng, Haohao Luo, Zhe Xu, Wei Liu, Jian Luan, Wanxia Cao, Ying Shen

HelpSteer: Multi-attribute Helpfulness Dataset for SteerLM (2024.naacl-long)

Challenge:	Large Language Models (LLMs) are sensitive to the contextual position of information in input.
Approach:	They introduce Attention-Driven Reranking (AttnRank) which estimates a model’s intrinsic positional attention preferences using a small calibration set and reorders retrieved documents or few-shot examples to align the most salient content with these high-attention positions.
Outcome:	Experiments on multi-hop QA and few-shot in-context learning tasks show that AttnRank achieves substantial improvements across 10 large language models of varying architectures and scales, without modifying model parameters or training procedures.

Copied to clipboard

Zhilin Wang, Yi Dong, Jiaqi Zeng, Virginia Adams, Makesh Narsimhan Sreedhar, Daniel Egert, Olivier Delalleau, Jane Scowcroft, Neel Kant, Aidan Swope, Oleksii Kuchaiev

Challenge:	Existing helpfulness preference datasets do not specify what makes some responses more helpful and others less helpful.
Approach:	They use a dataset that has annotated for correctness, coherence, complexity, and verbosity.
Outcome:	The dataset has annotations for correctness, coherence, complexity, and verbosity in addition to overall helpfulness of responses.

BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Models (2024.emnlp-main)

Copied to clipboard

Yi Zeng, Weiyu Sun, Tran Huynh, Dawn Song, Bo Li, Ruoxi Jia

OS Agents: A Survey on MLLM-based Agents for Computer, Phone and Browser Use (2025.acl-long)

Challenge:	Safety backdoors in large language models can be triggered while evading detection during normal interactions.
Approach:	They propose a bi-level optimization method that uses a key insight: backdoor triggers induce a uniform drift in the model’s embedding space . inner level identifies universal perturbations to the decoder’s embedded spaces that steer the model towards defender-defined unwanted behaviors; outer level fine-tunes the model to reinforce safe behaviors against these perturbations.
Outcome:	The proposed mitigation method reduces the success rate of safety backdoor attacks from over 95% to 1% for general harmful behaviors and from 47% to 0% for Sleeper Agents, without compromising the model’s usefulness.

Copied to clipboard

Xueyu Hu, Tao Xiong, Biao Yi, Zishu Wei, Ruixuan Xiao, Yurun Chen, Jiasheng Ye, Meiling Tao, Xiangxin Zhou, Ziyu Zhao, Yuhuai Li, Shengze Xu, Shenzhi Wang, Xinchen Xu, Shuofei Qiao, Zhaokai Wang, Kun Kuang, Tieyong Zeng, Liang Wang, Jiwei Li, Yuchen Eleanor Jiang, Wangchunshu Zhou, Guoyin Wang, Keting Yin, Zhou Zhao, Hongxia Yang, Fan Wu, Shengyu Zhang, Fei Wu

Asynchronous Convergence in Multi-Task Learning via Knowledge Distillation from Converged Tasks (2022.naacl-industry)

Challenge:	a new generation of (M)LLMs is enabling the creation of superintelligent AI assistants . OS Agents can complete tasks autonomously and have the potential to significantly enhance the lives of billions of users worldwide.
Approach:	They propose to build OS Agents that operate within operating systems' GUIs and GUIs . they examine evaluation metrics and benchmarks to identify promising directions .
Outcome:	The proposed agents are based on operating systems (OS) and operating systems frameworks.

Copied to clipboard

Weiyi Lu, Sunny Rajagopalan, Priyanka Nigam, Jaspreet Singh, Xiaodi Sun, Yi Xu, Belinda Zeng, Trishul Chilimbi

TiMem: Temporal-Hierarchical Memory Consolidation for Long-Horizon Conversational Agents (2026.findings-acl)

Challenge:	Multi-task learning (MTL) aims to solve multiple tasks by sharing a base representation among them.
Approach:	They propose an approach that allows for "asynchronous" convergence among the tasks where each task can converge on its own schedule.
Outcome:	The proposed method outperforms existing methods in two 5-task MTL setups.

Copied to clipboard

Kai Li, Xuanqing Yu, Ziyi Ni, Yi Zeng, Yao Xu, Zheqing Zhang, Xin Li, Jitao Sang, Xiaogang Duan, Xuelei Wang, Chengbao Liu, Jie Tan

Dialogue Systems & Conversational Agents

Challenge:	Existing memory frameworks provide limited support for temporally structured information across hierarchical levels, leading to fragmented memories and unstable long-horizon personalization.
Approach:	They propose a temporal–hierarchical memory framework that organizes conversations through a Temporal Memory Tree.
Outcome:	The proposed framework outperforms baselines while reducing the recalled memory length by 52.20%.

ReAugKD: Retrieval-Augmented Knowledge Distillation For Pre-trained Language Models (2023.acl-short)

Copied to clipboard

Jianyi Zhang, Aashiq Muhamed, Aditya Anantharaman, Guoyin Wang, Changyou Chen, Kai Zhong, Qingjun Cui, Yi Xu, Belinda Zeng, Trishul Chilimbi, Yiran Chen