Papers by Yan Pan
Towards Explainable Computerized Adaptive Testing with Large Language Model (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods focus on minimizing the number of questions required to assess ability, lacking clear and reliable explanations for the question selection process. |
| Approach: | They propose to use large language models to enhance computer adaptive testing (CAT) by providing human-like interpretability and explanations. |
| Outcome: | The proposed agent-based CAT performs comparably or superior to traditional CAT methods in accuracy and significantly improves student trust and satisfaction. |
DynaThink: Fast or Slow? A Dynamic Decision-Making Framework for Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have emerged as prominent foundation models for diverse applications due to their outstanding ability to understand and generate humanlike text. |
| Approach: | They propose a dynamic decision-making framework that categorizes tasks into two distinct pathways: 'Fast' and 'Slow' they propose 'self-consistency' strategy to replace the straight-forward decoding method used in COT prompting . |
| Outcome: | The proposed method achieves more than 3% increase in accuracy with lower cost on five popular reasoning benchmarks. |
Cool-Fusion: Fuse Large Language Models without Training (2025.acl-long)
Copied to clipboard
| Challenge: | Cool-Fusion is a simple yet effective approach to combine two or more heterogeneous large language models . |
| Approach: | They propose a method that fuses the knowledge of two or more heterogeneous large language models to leverage complementary strengths. |
| Outcome: | The proposed method increases accuracy from three strong source LLMs on GSM8K by 17.4%. |
Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning (2026.acl-long)
Copied to clipboard
Sikuan Yan, Xiufeng Yang, Zuchao Huang, Ercong Nie, Zifeng Ding, Zonggen Li, Xiaowen Ma, Jinhe Bi, Kristian Kersting, Jeff Z. Pan, Hinrich Schuetze, Volker Tresp, Yunpu Ma
| Challenge: | Large Language Models (LLMs) are stateless and limited by a finite context window, preventing them from maintaining knowledge across long conversations or evolving tasks. |
| Approach: | They propose a reinforcement learning framework that empowers LLMs to actively manage external memory through two specialized agents. |
| Outcome: | The proposed framework outperforms baselines and benchmarks across diverse question types, three benchmarks, and multiple model scales. |
MMedAgent: Learning to Use Medical Tools with Multi-modal Agent (2024.findings-emnlp)
Copied to clipboard
Binxu Li, Tiankai Yan, Yuanting Pan, Jie Luo, Ruiyang Ji, Jiayuan Ding, Zhe Xu, Shilong Liu, Haoyu Dong, Zihao Lin, Yixin Wang
| Challenge: | Multi-modal Large Language Models (MLLMs) exhibit limited generality and often fall short when compared to specialized models. |
| Approach: | They propose a multi-modal medical agent that picks the most suitable medical tools based on user inputs. |
| Outcome: | The proposed agent performs better than open-source models and the closed-source model, GPT-4o. |
MAIN-RAG: Multi-Agent Filtering Retrieval-Augmented Generation (2025.acl-long)
Copied to clipboard
Chia-Yuan Chang, Zhimeng Jiang, Vineeth Rakesh, Menghai Pan, Chin-Chia Michael Yeh, Guanchu Wang, Mingzhi Hu, Zhichao Xu, Yan Zheng, Mahashweta Das, Na Zou
| Challenge: | Existing RAG systems struggle with the quality of retrieval documents, causing performance degradation and reducing performance. |
| Approach: | They propose a training-free RAG framework that leverages multiple LLM agents to collaboratively filter and score retrieved documents. |
| Outcome: | The proposed framework outperforms existing RAG frameworks in QA benchmarks and shows superior answer consistency and answer accuracy over baseline methods. |
Enhancing Foundation Models in Transaction Understanding with LLM-based Sentence Embeddings (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing foundation models for tabular transactional data rely on index-based representations for categorical merchant fields. |
| Approach: | They propose a framework that uses LLM-generated embeddings as semantic initializations for lightweight transaction models. |
| Outcome: | The proposed framework improves performance on large transaction datasets. |
Thinking Before You Speak: A Proactive Test-time Scaling Approach (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models often exhibit deficiencies with complex reasoning tasks, such as maths, due to the discrepancy between human reasoning patterns and those presented in training data. |
| Approach: | They propose to insert insights between consecutive reasoning steps to bridge this gap by generating insights between the next reasoning steps. |
| Outcome: | Experiments on mathematical datasets confirm the effectiveness of the proposed reasoning framework on complex problems. |
TS-SQL: Test-driven Self-refinement for Text-to-SQL (2025.findings-emnlp)
Copied to clipboard
| Challenge: | null |
| Approach: | null |
| Outcome: | null |
CoVerRL: Breaking the Consensus Trap in Label-Free Reasoning via Generator-Verifier Co-Evolution (2026.acl-long)
Copied to clipboard
Teng Pan, Yuchen Yan, Zixuan Wang, Ruiqing Zhang, Guiyang Hou, Wenqi Zhang, Weiming Lu, Jun Xiao, Yongliang Shen
| Challenge: | Label-free reinforcement learning enables large language models to improve reasoning capabilities . but as training maximizes self-consistency, output diversity collapses, authors say . authors propose a framework where a single model alternates between generator and verifier roles . |
| Approach: | They propose a framework where a model alternates between generator and verifier roles, bootstrapping each other. |
| Outcome: | Experiments show that CoVerRL outperforms label-free baselines on reasoning benchmarks . the framework can be used to improve reasoning abilities without ground-truth supervision . |
ProBench: Judging Multimodal Foundation Models on Open-ended Multi-domain Expert Tasks (2025.findings-acl)
Copied to clipboard
| Challenge: | Solving expert-level multimodal tasks requires strong user query understanding, domain-specific knowledge, and advanced reasoning abilities. |
| Approach: | They propose a benchmark of open-ended user queries encapsulating professional expertise and advanced reasoning. |
| Outcome: | The proposed benchmark is publicly accessible at TBC. |
Towards Knowledge-Intensive Text-to-SQL Semantic Parsing with Formulaic Knowledge (2022.emnlp-main)
Copied to clipboard
Longxu Dou, Yan Gao, Xuqi Liu, Mingyang Pan, Dingzirui Wang, Wanxiang Che, Dechen Zhan, Min-Yen Kan, Jian-Guang Lou
| Challenge: | Existing approaches to text-to-SQL require domain knowledge to parse expert questions into SQL queries. |
| Approach: | They propose a framework to leverage domain knowledge during parsing by building a new benchmark KnowSQL with domain-specific questions. |
| Outcome: | The proposed framework improves the performance of the proposed benchmark by 28.2%. |
ASD-iLLM:An Intervention Large Language Model for Autistic Children based on Real Clinical Dialogue Intervention Dataset (2025.findings-emnlp)
Copied to clipboard
Shuzhong Lai, Chenxi Li, Junhong Lai, Yucun Zhong, Chenyu Yan, Xiang Li, Haifeng Li, Gang Pan, Lin Yao, Yueming Wang
| Challenge: | Currently, leveraging large language models (LLMs) for autism intervention is a significant yet challenging task, especially when directly employing LLMs as an intervention doctor. |
| Approach: | They propose a framework for training LLMs to conduct dialogue interventions in accordance with the principles of Applied Behavior Analysis (ABA) they also propose 'role-play' strategy in which LLM act as autistic children to comprehensively evaluate the doctor model's capabilities at the dialogue level. |
| Outcome: | The proposed framework outperforms existing models in both automatic and human evaluation, with intervention strategies and dialogue style more closely resembling those of clinical intervention doctors. |
Living in the Moment: Can Large Language Models Grasp Co-Temporal Reasoning? (2024.acl-long)
Copied to clipboard
| Challenge: | Current temporal reasoning datasets are limited to questions about single or isolated events, falling short in mirroring the realistic temporal characteristics involving concurrent nature and intricate temporal interconnections. |
| Approach: | They propose a co-temporal Question Answering benchmark that contains four co-time scenarios with 4,748 samples for evaluating the co-timing abilities of large language models. |
| Outcome: | The proposed benchmarks show that current LLMs struggle on CoTempQA tasks even when enhanced with Chain of Thought methodologies. |
Reading Like HER: Human Reading Inspired Extractive Summarization (D19-1)
Copied to clipboard
| Challenge: | Existing methods for extracting text summarization are abstractive and extractive. |
| Approach: | They propose a novel approach for extractive summarization by simulating two stages . they adopt a convolutional neural network to encode gist of paragraphs for rough reading . |
| Outcome: | The proposed method significantly outperforms the state-of-the-art extractive methods on CNN and DailyMail datasets. |
From Long to Lean: Performance-aware and Adaptive Chain-of-Thought Compression via Multi-round Refinement (2025.emnlp-main)
Copied to clipboard
| Challenge: | Chain-of-Thought reasoning introduces significant inference latency due to its verbosity. |
| Approach: | They propose a framework that leverages token elasticity phenomenon to progressively compress CoTs via multiround refinement. |
| Outcome: | The proposed method achieves an average accuracy improvement of 5.6% over state-of-the-art baselines while reducing CoT length by an average of 47 tokens and significantly lowering latency. |
Towards Efficient CoT Distillation: Self-Guided Rationale Selector for Better Performance with Fewer Rationales (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on rationale quality underestimates the importance of CoT distillation, focusing primarily on data quantity, which may result in transferring noisy or incorrect information to the student model. |
| Approach: | They propose a method which can discern and select high quality rationales for distillation and a Rationale Difficulty metric to measure the ability of the student model to generate the correct answer under a given rationale. |
| Outcome: | The proposed method achieves 4.6% accuracy improvement over baseline data on seven datasets over three tasks, controlling accuracy, diversity, and difficulty. |
MathFusion: Enhancing Mathematical Problem-solving of LLM through Instruction Fusion (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown impressive progress in mathematical problem-solving . current approaches to enhance mathematical reasoning focus on instance-level modifications . |
| Approach: | They propose a framework that enhances mathematical reasoning through cross-problem instruction synthesis. |
| Outcome: | The proposed framework boosts mathematical reasoning by 18.0 points while maintaining high data efficiency. |
Improving Open-Domain Dialogue Systems via Multi-Turn Incomplete Utterance Restoration (D19-1)
Copied to clipboard
| Challenge: | Experimental results show that restoring incomplete utterances from context improves the performance of open-domain dialogue systems. |
| Approach: | They propose to use a dataset to restore incomplete utterances from context . they propose to pick and combine the data to restore the incomplete . |
| Outcome: | The proposed model significantly boosts response quality of open-domain dialogue systems. |
Single-Pass, Depth-Selective Reading for Multi-Aspect Sentiment Analysis (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models re-encode the sentence for each aspect or rely on static use of deep representations, leading to redundant computation and limited adaptivity. |
| Approach: | They propose a single-pass inference framework that encodes each sentence once to construct a reusable, depth-ordered substrate. |
| Outcome: | Experiments show that DABS reduces end-to-end computation by 60% in multi-aspect settings. |
Demystify the Role of Memory in Machine Learning Engineering Agents (2026.findings-acl)
Copied to clipboard
Xinyu Zhao, Junpeng Wang, Yuzhong Chen, Menghai Pan, Chin-Chia Michael Yeh, Jiarui Sun, Yan Zheng, Mahashweta Das, Tianlong Chen
| Challenge: | Unlike short, reactive exchanges, MLE agents solve tasks through cycles of experimentation and improvement where past errors can inform future success. |
| Approach: | They propose a dynamic coding memory that captures and reuses debugging experiences and integrates it into two representative agent paradigms. |
| Outcome: | The proposed agent model captures and reuses debugging experiences and integrates it into two agent paradigms. |
HoH: A Dynamic Benchmark for Evaluating the Impact of Outdated Information on Retrieval-Augmented Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Current approaches to addressing knowledge outdating in LLMs struggle with retrieval and generation aspects when handling outdated information. |
| Approach: | They propose a benchmark to evaluate the impact of outdated information on RAG . they use token-level diff algorithms and LLM pipelines to create a large-scale QA dataset . |
| Outcome: | The proposed benchmark analyzes the impact of outdated information on RAG performance. |
Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub (2025.acl-long)
Copied to clipboard
Bohan Lyu, Xin Cong, Heyang Yu, Pan Yang, Cheng Qian, Zihe Wang, Yujia Qin, Yining Ye, Yaxi Lu, Chen Qian, Zhong Zhang, Yukun Yan, Yankai Lin, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing approaches lack flexibility to address diverse and ever-evolving user queries in open domains. |
| Approach: | They propose to evaluate LLMs on open-domain knowledge that requires tools to solve diverse and ever-evolving user queries. |
| Outcome: | The proposed system outperforms baselines in the open domain task-solving benchmark. |
Reward Generalization in RLHF: A Topological Perspective (2025.findings-acl)
Copied to clipboard
Tianyi Alex Qiu, Fanzhi Zeng, Jiaming Ji, Dong Yan, Kaile Wang, Jiayi Zhou, Yang Han, Josef Dai, Xuehai Pan, Yaodong Yang
| Challenge: | Existing alignment methods share a common topology of information flow, but their alternatives have not been thoroughly explored. |
| Approach: | They propose a theory of reward generalization in reinforcement learning from human feedback . they propose induced Bayesian networks to model the impact of dataset topologies on reward generalisation . |
| Outcome: | The proposed method achieves an average win rate of 65% on three NLP tasks. |