Papers by Qingyao Ai
SelfRACG: Enabling LLMs to Self-Express and Retrieve for Code Generation (2025.emnlp-main)
Copied to clipboard
Qian Dong, Jia Chen, Qingyao Ai, Hongning Wang, Haitao Li, null Yiwu, Yao Hu, Yiqun Liu, Shaoping Ma
| Challenge: | Existing retrieval-augmented code generation methods fail to accurately fetch the knowledge required for code generation for consecutive code fragments. |
| Approach: | They propose a paradigm that enables large language models to Self-express their information needs to enhance retrieval-augmented code generation methods. |
| Outcome: | Experiments show that SelfRACG can retrieve external knowledge that better aligns with the LLM’s own information needs, resulting in superior generation performance compared to vanilla RACG. |
PEPE: Long-context Extension for Large Language Models via Periodic Extrapolation Positional Encodings (2025.findings-emnlp)
Copied to clipboard
Jikun Hu, Dongsheng Guo, Yuli Liu, Qingyao Ai, Lixuan Wang, Xuebing Sun, Qilei Zhang, Quan Zhou, Cheng Luo
| Challenge: | Long-context extension attempts to extend contextual window in pre-trained LLMs . primary method involves expanding initial positional encodings, disrupting positional learning . |
| Approach: | They propose a new extension strategy based on Rotary Position Embedding to extend contextual window in pre-trained large language models. |
| Outcome: | The proposed method can extend the contextual window in pre-trained large language models . expansion disrupts positional encodings learned during pre-training, authors show . |
Beyond Experience Retrieval: Learning to Generate Utility-Optimized Structured Experience for Frozen LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are largely static and often redo reasoning or repeat mistakes. Prior experience reuse relies on external retrieval, which is similarity-based, can introduce noise, and adds latency. |
| Approach: | They propose a lightweight plug-in that stores experience in its parameters and generates a structured, instance-tailored experience entry in a single forward pass to guide a frozen LLM executor. |
| Outcome: | Experiments on mathematical reasoning benchmarks show consistent accuracy gains across executors with low overhead. |
SimVBG: Simulating Individual Values by Backstory Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have strong human-like capabilities, but rarely simulating individualized human values. |
| Approach: | They propose a framework that simulates individual values based on individual backstories . they use structured data on an individual to transform their backstoried information to a backstory . |
| Outcome: | The proposed framework improves top-1 accuracy by more than 10% over retrieval-augmented generation methods. |
CaseEncoder: A Knowledge-enhanced Pre-trained Model for Legal Case Encoding (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing legal-oriented PLMs rely on replacing general domain training data with legal data or extending the input length to fit the long-length characteristic of legal data. |
| Approach: | They propose a legal document encoder that leverages fine-grained legal knowledge in both the data sampling and pre-training phases. |
| Outcome: | The proposed model outperforms existing general domain pre-training models and legal-specific pre-trainers on multiple benchmarks. |
STARD: A Chinese Statute Retrieval Dataset Derived from Real-life Queries by Non-professionals (2024.findings-emnlp)
Copied to clipboard
Weihang Su, Yiran Hu, Anzhe Xie, Qingyao Ai, Quezi Bing, Ning Zheng, Yun Liu, Weixing Shen, Yiqun Liu
| Challenge: | Existing statute retrieval benchmarks emphasize formal and professional queries from sources like bar exams and legal case documents . existing retrieval approaches that lack domain-specific knowledge may struggle to capture the meanings of specialized terms accurately. |
| Approach: | They propose a dataset that captures the complexity and diversity of real queries from the general public. |
| Outcome: | The proposed dataset captures the complexity and diversity of real queries from the general public. |
DRAGIN: Dynamic Retrieval Augmented Generation based on the Real-time Information Needs of Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing dynamic RAG methods fail to address the information needs of large language models (LLMs) despite their impressive capabilities, these models often produce text that seems coherent and plausible but factually incorrect, a problem commonly known as hallucination. |
| Approach: | They propose a dynamic retrieval augmented generation paradigm that actively decides when and what to retrieve during the text generation process of Large Language Models. |
| Outcome: | The proposed framework achieves superior performance over 4 knowledge-intensive generation datasets. |
Augmenting Multi-Agent Communication with State Delta Trajectory (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multi-agent systems based on large language models (LLMs) have shown to be effective in downstream tasks. |
| Approach: | They propose a protocol that transfers both natural language tokens and token-wise state transition trajectory from one agent to another. |
| Outcome: | The proposed protocol can transfer both natural language tokens and token-wise state transition trajectory from one agent to another. |
TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona Simulation (2026.findings-acl)
Copied to clipboard
Bangde Du, Minghao Guo, Songming He, Ziyi Ye, Xi Zhu, Weihang Su, Shuqi Zhu, Yujia Zhou, Yongfeng Zhang, Qingyao Ai, Yiqun Liu
| Challenge: | Existing studies show that advanced LLMs produce text indistinguishable from human writing. |
| Approach: | They propose a benchmark to assess persona simulation across diverse contexts by decomposing the evaluation into six fundamental capabilities including opinion consistency, memory recall, logical reasoning, persona tone, and syntactic style. |
| Outcome: | The proposed model achieves moderate accuracy but falls short of the basic capabilities needed to simulate personas in real-world contexts. |
CalibraEval: Calibrating Prediction Distribution to Mitigate Selection Bias in LLMs-as-Judges (2025.acl-long)
Copied to clipboard
| Challenge: | Empirical evaluations of large language models demonstrate that they improve performance in a wide range of tasks. |
| Approach: | They propose a label-free method for mitigating selection bias during inference by reformulating debiasing as an optimization task. |
| Outcome: | The proposed method mitigates selection bias and improves performance compared to existing methods. |
Unsupervised Real-Time Hallucination Detection based on the Internal States of Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on hallucination detection for LLMs focus on how to identify possible factrelated errors in outputs. |
| Approach: | They propose an unsupervised training framework that leverages the internal states of LLMs for real-time hallucination detection without requiring manual annotations. |
| Outcome: | The proposed framework outperforms existing state-of-the-art methods in hallucination detection. |
LegalAgentBench: Evaluating LLM Agents in Legal Domain (2025.acl-long)
Copied to clipboard
Haitao Li, Junjie Chen, Jingli Yang, Qingyao Ai, Wei Jia, Youfeng Liu, Kai Lin, Yueyue Wu, Guozhi Yuan, Yiran Hu, Wuyue Wang, Yiqun Liu, Minlie Huang
| Challenge: | Existing general-domain benchmarks do not capture complexity of real-world judicial cognition and decision-making. |
| Approach: | They propose a benchmark specifically designed to evaluate LLM Agents in the legal domain. |
| Outcome: | The proposed benchmark includes 17 corpora from real-world legal scenarios and provides 37 tools for interacting with external knowledge. |
Chinese Court Simulation with LLM-Based Agents System (2026.findings-acl)
Copied to clipboard
Kaiyuan Zhang, Jiaqi Li, Yueyue Wu, Haitao Li, Cheng Luo, Shaokun Zou, Yujia Zhou, Weihang Su, Yiqun Liu, Qingyao Ai
| Challenge: | Existing studies have neglected the systematic design and procedure evaluation of court simulations, which are critical to the credibility and usage of court simulators in practice. |
| Approach: | They propose a court simulation paradigm based on the real-world procedure structure of Chinese courts and a framework that focuses on both legal judgment prediction and court procedure analysis. |
| Outcome: | The proposed model outperforms judges and lawyers from the real trials in many aspects. |
Decoupling Reasoning and Knowledge Injection for In-Context Knowledge Editing (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge editing approaches directly edit model context without isolating target knowledge from the reasoning path of model inference, resulting in unreliable and low-quality outputs, especially in multi-hop tasks. |
| Approach: | They propose a framework that separates model reasoning from knowledge editing and propose 'DecKER' that allows users to modify specific factual associations without retraining the entire model. |
| Outcome: | The proposed framework significantly improves multi-hop reasoning performance by mitigating knowledge conflicts and preserving reasoning integrity. |
Prompt Refinement with Image Pivot for Text-to-Image Generation (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in text-to-image generation have markedly expanded the boundaries of digital artistry, enabling the creation of visually compelling images with unprecedented ease. |
| Approach: | They propose to decompose the prompt refinement process into two tasks: inferring user-preferred images from user languages and translating them into system languages. |
| Outcome: | Experiments show that PRIP outperforms baselines and transfers to unseen systems in a zero-shot manner. |
Knowledge Editing through Chain-of-Thought (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge editing methods focus on multi-hop QA tasks and require frequent retraining. |
| Approach: | They propose a new knowledge editing framework that updates large language models with new information to maintain their world knowledge without retraining. |
| Outcome: | The proposed method achieves state-of-the-art performance while offering superior generalization, effectiveness, and stability compared to existing methods. |