Papers by Yujia Zhou

Exploring Mode Connectivity for Pre-trained Language Models (2022.emnlp-main)

Challenge:	End-to-end generative retrieval models produce document identifiers in response to a query . however, this approach has two challenges: an overemphasis on top-1 results at the expense of overall ranking quality.
Approach:	They propose a generative retrieval model with reinforcement learning from relevance feedback to align token-level docid generation with document-level relevance estimation.
Outcome:	The proposed model aligns token-level docid generation with document-level relevance estimation.

Copied to clipboard

Yujia Qin, Cheng Qian, Jing Yi, Weize Chen, Yankai Lin, Xu Han, Zhiyuan Liu, Maosong Sun, Jie Zhou

Knowledge Inheritance for Pre-trained Language Models (2022.naacl-main)

Challenge:	Recent years have witnessed the prevalent application of pre-trained language models (PLMs) in NLP. From the perspective of parameter space, PLMs provide generic initialization, starting from which high-performance minima could be found.
Approach:	They investigate the geometric connections of different minima through the lens of mode connectivity, which measures whether two minima can be connected with a low-loss path.
Outcome:	The proposed model can be used to find low-loss paths between two minima, and to understand how their mode connectivity affects their task knowledge.

Copied to clipboard

Yujia Qin, Yankai Lin, Jing Yi, Jiajie Zhang, Xu Han, Zhengyan Zhang, Yusheng Su, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

ProQA: Structural Prompt-based Pre-training for Unified Question Answering (2022.naacl-main)

Challenge:	Existing large-scale pre-trained language models are mainly trained from scratch individually, ignoring that many well-taught PLMs are available.
Approach:	They propose a pre-training framework called knowledge inheritance and propose auxiliary supervision to efficiently learn larger PLMs.
Outcome:	The proposed framework can be used to train large-scale language models with huge parameters and a large dataset can be adapted to domain adaptation and knowledge transfer.

Copied to clipboard

Wanjun Zhong, Yifan Gao, Ning Ding, Yujia Qin, Zhiyuan Liu, Ming Zhou, Jiahai Wang, Jian Yin, Nan Duan

An Element is Worth a Thousand Words: Enhancing Legal Case Retrieval by Incorporating Legal Elements (2024.findings-acl)

Challenge:	Existing QA research on question answering is focused on specific question types, knowledge domains, or reasoning skills.
Approach:	They propose a unified QA paradigm that solves various tasks through a single model.
Outcome:	The proposed model improves QA-centric ability on 11 QA benchmarks.

Copied to clipboard

Chenlong Deng, Zhicheng Dou, Yujia Zhou, Peitian Zhang, Kelong Mao

Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents (2024.acl-long)

Challenge:	Existing methods for legal case retrieval lack the definition of relevance for legal cases . however, the definition goes beyond the common semantic relevance of ad-hoc retrieval.
Approach:	They propose a legal element dataset that incorporates legal elements into a semi-automatic method . they propose two models to enhance legal search using legal elements .
Outcome:	The proposed models outperform existing methods in enhancing legal search using legal elements.

Copied to clipboard

Cheng Qian, Bingxiang He, Zhong Zhuang, Jia Deng, Yujia Qin, Xin Cong, Zhong Zhang, Jie Zhou, Yankai Lin, Zhiyuan Liu, Maosong Sun

RetroLLM: Empowering Large Language Models to Retrieve Fine-grained Evidence within Generation (2025.acl-long)

Challenge:	Current language model-driven agents lack mechanisms for effective user participation, which is crucial given the vagueness commonly found in user instructions.
Approach:	They propose a benchmark to inspect users’ implicit intentions through explicit queries and a model expert as the upstream in agent design to enhance user-agent interaction.
Outcome:	The proposed approach excels at identifying vague user tasks, recovering and summarizing critical missing information, setting precise and necessary agent execution goals, and minimizing redundant tool usage, thus boosting overall efficiency.

Copied to clipboard

Xiaoxi Li, Jiajie Jin, Yujia Zhou, Yongkang Wu, Zhonghua Li, Ye Qi, Zhicheng Dou

BIDER: Bridging Knowledge Inconsistency for Efficient Retrieval-Augmented LLMs via Key Supporting Evidence (2024.findings-acl)

Challenge:	Existing methods rely on separate retrievers to fetch top-k text chunks for generating evidence, and they lack joint optimization.
Approach:	They propose a framework that integrates retrieval and generation into a single, auto-regressive process, enabling LLMs to directly generate fine-grained evidence from the corpus with constrained decoding.
Outcome:	Extensive experiments on five open-domain QA datasets demonstrate the proposed framework’s superior performance across both in-domain and out-of-domain tasks.

Copied to clipboard

Jiajie Jin, Yutao Zhu, Yujia Zhou, Zhicheng Dou

Green & Sustainable NLP

Challenge:	Large language models (LLMs) have demonstrated efficacy in knowledge-intensive tasks such as open-domain QA, but inconsistencies between retrieval knowledge and the necessary knowledge for LLMs, leading to a decline in LLM’s answer quality.
Approach:	They propose a retrieval-augmented large language model that refines retrieval documents into Key Supporting Evidence (KSE) through knowledge synthesis, supervised fine-tuning, and preference alignment.
Outcome:	The proposed model outperforms existing methods in five datasets and boosts answer quality by 7% while reducing input content length by 80%.

K-PLUG: Knowledge-injected Pre-trained Language Model for Natural Language Understanding and Generation in E-Commerce (2021.findings-emnlp)

Copied to clipboard

Song Xu, Haoran Li, Peng Yuan, Yujia Wang, Youzheng Wu, Xiaodong He, Ying Liu, Bowen Zhou

Beyond Experience Retrieval: Learning to Generate Utility-Optimized Structured Experience for Frozen LLMs (2026.acl-long)

Challenge:	Existing pre-trained language models are not explicitly aware of domain-specific knowledge, which is essential for downstream tasks in many domains, such as tasks in e-commerce scenarios.
Approach:	They propose a knowledge-injected pre-trained language model that can be transferred to both natural language understanding and generation tasks.
Outcome:	The proposed model significantly outperforms baselines across the board in e-commerce scenarios.

Copied to clipboard

Xuancheng Li, Haitao Li, Yujia Zhou, Yiqun Liu, Qingyao Ai

SimVBG: Simulating Individual Values by Backstory Generation (2025.emnlp-main)

Challenge:	Large language models (LLMs) are largely static and often redo reasoning or repeat mistakes. Prior experience reuse relies on external retrieval, which is similarity-based, can introduce noise, and adds latency.
Approach:	They propose a lightweight plug-in that stores experience in its parameters and generates a structured, instance-tailored experience entry in a single forward pass to guide a frozen LLM executor.
Outcome:	Experiments on mathematical reasoning benchmarks show consistent accuracy gains across executors with low overhead.

Copied to clipboard

Bangde Du, Ziyi Ye, Zhijing Wu, Monika A. Jankowska, Shuqi Zhu, Qingyao Ai, Yujia Zhou, Yiqun Liu

Challenge:	Large language models (LLMs) have strong human-like capabilities, but rarely simulating individualized human values.
Approach:	They propose a framework that simulates individual values based on individual backstories . they use structured data on an individual to transform their backstoried information to a backstory .
Outcome:	The proposed framework improves top-1 accuracy by more than 10% over retrieval-augmented generation methods.

Search-o1: Agentic Search-Enhanced Large Reasoning Models (2025.emnlp-main)

Copied to clipboard

Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, Zhicheng Dou

Reasoning

Challenge:	Large reasoning models (LRMs) have demonstrated impressive long stepwise reasoning capabilities through large-scale reinforcement learning.
Approach:	They propose a framework that enhances large reasoning models with an agentic retrieval-augmented generation mechanism and a Reason-in-Documents module for refining retrieved documents.
Outcome:	The proposed framework enhances LRMs with an agentic retrieval-augmented generation mechanism and Reason-in-Documents module for refining retrieved documents.

Grounding Language Model with Chunking-Free In-Context Retrieval (2024.acl-long)

Copied to clipboard

Hongjin Qian, Zheng Liu, Kelong Mao, Yujia Zhou, Zhicheng Dou

Responsible & Trustworthy NLP

Challenge:	CFIC retrieval approach eliminates the need for document chunking and provides a more efficient and efficient method for RAG systems.
Approach:	They propose a Chunking-Free In-Context retrieval approach specifically tailored for RAG systems . they employ auto-aggressive decoding to accurately identify specific evidence text .
Outcome:	The proposed method is better than traditional methods on open question answering datasets.

Augmenting Multi-Agent Communication with State Delta Trajectory (2025.emnlp-main)

Copied to clipboard

Yichen Tang, Weihang Su, Yujia Zhou, Yiqun Liu, Min Zhang, Shaoping Ma, Qingyao Ai

Reasoning

Challenge:	Multi-agent systems based on large language models (LLMs) have shown to be effective in downstream tasks.
Approach:	They propose a protocol that transfers both natural language tokens and token-wise state transition trajectory from one agent to another.
Outcome:	The proposed protocol can transfer both natural language tokens and token-wise state transition trajectory from one agent to another.

TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona Simulation (2026.findings-acl)

Copied to clipboard

Bangde Du, Minghao Guo, Songming He, Ziyi Ye, Xi Zhu, Weihang Su, Shuqi Zhu, Yujia Zhou, Yongfeng Zhang, Qingyao Ai, Yiqun Liu

Natural Language Interfaces

Challenge:	Existing studies show that advanced LLMs produce text indistinguishable from human writing.
Approach:	They propose a benchmark to assess persona simulation across diverse contexts by decomposing the evaluation into six fundamental capabilities including opinion consistency, memory recall, logical reasoning, persona tone, and syntactic style.
Outcome:	The proposed model achieves moderate accuracy but falls short of the basic capabilities needed to simulate personas in real-world contexts.

CalibraEval: Calibrating Prediction Distribution to Mitigate Selection Bias in LLMs-as-Judges (2025.acl-long)

Copied to clipboard

Haitao Li, Junjie Chen, Qingyao Ai, Zhumin Chu, Yujia Zhou, Qian Dong, Yiqun Liu

Robustness in NLP

Challenge:	Empirical evaluations of large language models demonstrate that they improve performance in a wide range of tasks.
Approach:	They propose a label-free method for mitigating selection bias during inference by reformulating debiasing as an optimization task.
Outcome:	The proposed method mitigates selection bias and improves performance compared to existing methods.

How Credible Is an Answer From Retrieval-Augmented LLMs? Investigation and Evaluation With Multi-Hop QA (2025.coling-main)

Copied to clipboard

Yujia Zhou, Zheng Liu, Zhicheng Dou

Natural Language Interfaces

Challenge:	Retrieval-augmented large language models (RaLLMs) are reshaping knowledge acquisition, offering long-form, knowledge-grounded answers through advanced reasoning and generation capabilities.
Approach:	They propose a benchmarking system to evaluate RaLLMs' correctness and Groundedness to determine their reliability in multi-hop question-answering tasks.
Outcome:	The proposed model-based evaluation pipeline for multi-hop question-answering tasks reveals that the model generates inaccuracies when dealing with flawed or partial knowledge.

On Transferability of Prompt Tuning for Natural Language Processing (2022.naacl-main)

Copied to clipboard

Yusheng Su, Xiaozhi Wang, Yujia Qin, Chi-Min Chan, Yankai Lin, Huadong Wang, Kaiyue Wen, Zhiyuan Liu, Peng Li, Juanzi Li, Lei Hou, Maosong Sun, Jie Zhou

Responsible & Trustworthy NLP

Challenge:	Pre-trained language models (PLMs) can achieve comparable performance to full-parameter fine-tuning by tuning a few soft prompts, but require much more training time than fine-timing.
Approach:	They empirically investigate the transferability of soft prompts across different downstream tasks and PLMs to determine what decides prompt transferability.
Outcome:	The proposed method can achieve comparable performance to full-parameter fine-tuning by tuning a few soft prompts, but requires much more training time than fine-timing.

Unsupervised Real-Time Hallucination Detection based on the Internal States of Large Language Models (2024.findings-acl)

Copied to clipboard

Weihang Su, Changyue Wang, Qingyao Ai, Yiran Hu, Zhijing Wu, Yujia Zhou, Yiqun Liu

Responsible & Trustworthy NLP

Challenge:	Existing studies on hallucination detection for LLMs focus on how to identify possible factrelated errors in outputs.
Approach:	They propose an unsupervised training framework that leverages the internal states of LLMs for real-time hallucination detection without requiring manual annotations.
Outcome:	The proposed framework outperforms existing state-of-the-art methods in hallucination detection.

WebCPM: Interactive Web Search for Chinese Long-form Question Answering (2023.acl-long)

Copied to clipboard

Yujia Qin, Zihan Cai, Dian Jin, Lan Yan, Shihao Liang, Kunlun Zhu, Yankai Lin, Xu Han, Ning Ding, Huadong Wang, Ruobing Xie, Fanchao Qi, Zhiyuan Liu, Maosong Sun, Jie Zhou

ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning (2021.acl-long)

Challenge:	Long-form question answering requires two procedures: information retrieval and information synthesis.
Approach:	They propose a Chinese long-form question answering dataset called WebCPM . the dataset is based on a web search interface that engages with a search engine in real time .
Outcome:	The proposed dataset generates answers that are no worse than human-written ones . the dataset is the first Chinese LFQA dataset .

Copied to clipboard

Yujia Qin, Yankai Lin, Ryuichi Takanobu, Zhiyuan Liu, Peng Li, Heng Ji, Minlie Huang, Maosong Sun, Jie Zhou

Ambiguity Awareness Optimization: Towards Semantic Disambiguation for Direct Preference Optimization (2025.emnlp-main)

Challenge:	Existing pre-training objectives do not explicitly model relational facts in text . Experimental results show that ERICA can improve typical PLMs on several language understanding tasks, including relation extraction, entity typing and question answering.
Approach:	They propose a contrastive learning framework ERICA to obtain a deep understanding of entities and relations in text.
Outcome:	The proposed framework can improve PLMs on several language understanding tasks, especially under low-resource settings.

Copied to clipboard

Jian Li, Shenglin Yin, Yujia Zhang, Alan Zhao, Xi Chen, Xiaohui Zhou, Pengfei Xu

Learn to Copy from the Copying History: Correlational Copy Network for Abstractive Summarization (2021.emnlp-main)

Challenge:	Direct Preference Optimization (DPO) is a widely used reinforcement learning from human feedback (RLHF) method across various domains.
Approach:	They propose an approach that automatically re-weights ambiguous content to reduce ambiguities by calculating semantic similarity from preference pairs.
Outcome:	The proposed approach outperforms state-of-the-art approaches in performance across multiple model scales and widely adopted benchmark datasets.

Copied to clipboard

Haoran Li, Song Xu, Peng Yuan, Yujia Wang, Youzheng Wu, Xiaodong He, Bowen Zhou

Text Generation

Challenge:	Existing methods for abstractive summarization use encoder-decoder attention, but this leads to incomplete copying.
Approach:	They propose a copying scheme that takes advantage of prior copying distributions and explicitly encourages the model to copy the input word that is relevant to the previously copied one.
Outcome:	The proposed scheme achieves state-of-the-art on summarization benchmarks . it takes advantage of prior copying distributions and explicitly encourages copying .

Enhancing Chat Language Models by Scaling High-quality Instructional Conversations (2023.emnlp-main)

Copied to clipboard

Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Shengding Hu, Zhiyuan Liu, Maosong Sun, Bowen Zhou

Chinese Court Simulation with LLM-Based Agents System (2026.findings-acl)

Challenge:	a recent study validates the effectiveness of chat language models by fine-tuning instruction data.
Approach:	They propose to use a large-scale dataset of instructional conversations to fine-tune a conversational model on instruction data.
Outcome:	The proposed model outperforms open-source models in key metrics including scale, average length, diversity, coherence, etc.

Copied to clipboard

Kaiyuan Zhang, Jiaqi Li, Yueyue Wu, Haitao Li, Cheng Luo, Shaokun Zou, Yujia Zhou, Weihang Su, Yiqun Liu, Qingyao Ai

Different Tunes Played with Equal Skill: Exploring a Unified Optimization Subspace for Parameter-Efficient Tuning (2022.findings-emnlp)

Challenge:	Existing studies have neglected the systematic design and procedure evaluation of court simulations, which are critical to the credibility and usage of court simulators in practice.
Approach:	They propose a court simulation paradigm based on the real-world procedure structure of Chinese courts and a framework that focuses on both legal judgment prediction and court procedure analysis.
Outcome:	The proposed model outperforms judges and lawyers from the real trials in many aspects.

Copied to clipboard

Jing Yi, Weize Chen, Yujia Qin, Yankai Lin, Ning Ding, Xu Han, Zhiyuan Liu, Maosong Sun, Jie Zhou

Green & Sustainable NLP

Challenge:	Existing delta tuning algorithms freeze most of the parameters and only optimize minimal adaptive parameters.
Approach:	They propose to decompose DETs into a unified optimization subspace and conduct optimization within the subspace.
Outcome:	The proposed DETs achieve comparable performance to the original DET and can be transferred to another DET with non-trivial performance.

Decoupling Reasoning and Knowledge Injection for In-Context Knowledge Editing (2025.findings-acl)

Copied to clipboard

Changyue Wang, Weihang Su, Qingyao Ai, Yujia Zhou, Yiqun Liu

Reasoning

Challenge:	Existing knowledge editing approaches directly edit model context without isolating target knowledge from the reasoning path of model inference, resulting in unreliable and low-quality outputs, especially in multi-hop tasks.
Approach:	They propose a framework that separates model reasoning from knowledge editing and propose 'DecKER' that allows users to modify specific factual associations without retraining the entire model.
Outcome:	The proposed framework significantly improves multi-hop reasoning performance by mitigating knowledge conflicts and preserving reasoning integrity.

ELLE: Efficient Lifelong Pre-training for Emerging Data (2022.findings-acl)

Copied to clipboard

Yujia Qin, Jiajie Zhang, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

AIMMerging: Adaptive Iterative Model Merging Using Training Trajectories for Language Model Continual Learning (2025.emnlp-main)

Challenge:	Existing pre-trained language models are typically trained with static data, ignoring that streaming data of various sources may continuously grow.
Approach:	They propose to use function preserved model expansion to expand existing PLM's width and depth to improve efficiency of knowledge acquisition.
Outcome:	The proposed model improves pre-training efficiency and performance over existing models on BERT and GPT.

Copied to clipboard

Yujie Feng, Jian Li, Xiaoyu Dong, Pengfei Xu, Xiaohui Zhou, Yujia Zhang, Zexin Lu, Yasha Wang, Alan Zhao, Xu Chu, Xiao-Ming Wu