Papers by Yi Sui
Enabling Self-Improving Agents to Learn at Test Time With Human-In-The-Loop Guidance (2025.emnlp-industry)
Copied to clipboard
Yufei He, Ruoyu Li, Alex Chen, Yue Liu, Yulin Chen, Yuan Sui, Cheng Chen, Yi Zhu, Luca Luo, Frank Yang, Bryan Hooi
| Challenge: | Existing large language model (LLM) agents are unable to adapt to changing domain knowledge and rules. |
| Approach: | They propose an LLM agent framework that continuously learns updated domain knowledge at test time. |
| Outcome: | The proposed agent improves on a customer due diligence name screening task on . the agent learns updated domain knowledge at test time. |
Bridging External and Parametric Knowledge: Mitigating Hallucination of LLMs with Shared-Private Semantic Synergy in Dual-Stream Knowledge (2025.emnlp-main)
Copied to clipboard
| Challenge: | Retrieval-augmented generation (RAG) aims to mitigate the hallucination of Large Language Models (LLMs) however, external knowledge may contain noise and conflict with parametric knowledge of LLMs, leading to degraded performance. |
| Approach: | They propose a Dual-Stream Knowledge-Augmented Framework for Shared-Private Semantic Synergy that refines the traditional self-attention into a mixed-attention that distinguishes shared and private semantics for a controlled knowledge integration. |
| Outcome: | Extensive experiments show that the proposed framework achieves a superior performance over baselines. |
Think Less, Know More: State-Aware Reasoning Compression with Knowledge Guidance for Efficient Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing CoT compression methods struggle to balance accuracy and efficiency . long CoT reasoning also introduces an overthinking phenomenon, authors say . |
| Approach: | They propose a framework that performs step-wise CoT compression by modeling stage-specific redundancy sources and integrating with a retrieval-augmented guidance. |
| Outcome: | The proposed framework reduces average response length by 59.9% while improving accuracy by 4.8 points over existing methods. |
PLAWBENCH: A Rubric-Based Benchmark for Evaluating LLMs in Real-World Legal Practice (2026.acl-long)
Copied to clipboard
Yuzhen Shi, Huanghai Liu, Yiran HU, Song Gaojie, Xu Xinran, Yubo Ma, Tianyi Tang, Li Zhang, Qingjing Chen, Feng Di, Wenbo Lv, Weiheng Wu, Kexin Yang, Sen Yang, Wei Wang, Rongyao Shi, Qiu Yuanyang, Yuemeng Qi, Zhang Jingwen, Sui Xiaoyu, Yifan Chen, Zhang Yi, An Yang, Bowen Yu, Dayiheng Liu, Junyang Lin, Weixing Shen, Bing Zhao, Charles L. A. Clarke, HU Wei
| Challenge: | Existing benchmarks for large language models (LLMs) are coarse, single-dimensional metrics and do not explicitly assess fine-grained legal reasoning. |
| Approach: | They propose a Practical Law Benchmark to evaluate large language models in real-world legal practice scenarios. |
| Outcome: | The proposed model is based on 850 questions and 13 scenarios with expert-designed evaluation rubrics. |
Classifying and Addressing the Diversity of Errors in Retrieval-Augmented Generation Systems (2026.eacl-long)
Copied to clipboard
Kin Kwan Leung, Mouloud Belbahri, Yi Sui, Alex Labach, Xueying Zhang, Stephen Anthony Rose, Jesse C. Cresswell
| Challenge: | Existing work on RAG errors has not accounted for the complexity of real-world RAG systems and their failure modes. |
| Approach: | They propose a taxonomy of error types that can occur in realistic RAG systems and an auto-evaluation method that can be used to track errors during development. |
| Outcome: | The proposed method can be used in practice to track and address errors during development. |
A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing work on pre-trained generative models often fails to detect non-existent or incorrect content . Existing studies have attempted to detect hallucinations based on oracle references . |
| Approach: | They propose a token-level, reference-free hallucination detection task based on Wikipedia annotations to detect non-existent or incorrect content. |
| Outcome: | The proposed task is token-level, reference-free hallucination detection task and dataset . authors argue that the proposed task can be used in real-time to detect hallucines . |
HistLens: Mapping Idea Change across Concepts and Corpora (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to diachronic semantics and discourse analysis focus on a single concept or corpus, argues a new paper. |
| Approach: | They propose a framework for multi-concept, multi-corpus conceptual-history analysis that decomposes concept representations into interpretable features and tracks activation dynamics over time and across sources. |
| Outcome: | The proposed framework decomposes concept representations into interpretable features and tracks their activation dynamics over time and across sources. |