Papers by Tao Luo
OOP: Object-Oriented Programming Evaluation Benchmark for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | None Large language models (LLMs) are emerging as a key tool for automated programming. |
| Approach: | They compare performance of None Large language models with language understanding models on functional programming and object-oriented programming benchmarks. |
| Outcome: | The models perform relatively well on functional programming (FP) and object-oriented programming (OOP) benchmarks, while exhibiting poor performance on OOP benchmarks. |
Data Interpreter: An LLM Agent for Data Science (2025.findings-acl)
Copied to clipboard
Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Binhao Wu, Ceyao Zhang, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, Li Zhang, Lingyao Zhang, Min Yang, Mingchen Zhuge, Taicheng Guo, Tuo Zhou, Wei Tao, Robert Tang, Xiangtao Lu, Xiawu Zheng, Xinbing Liang, Yaying Fei, Yuheng Cheng, Yongxin Ni, Zhibin Gou, Zongze Xu, Yuyu Luo, Chenglin Wu
| Challenge: | Large Language Models (LLMs) excel in various domains but face challenges when applied to data science workflows due to their complex, multi-stage nature. |
| Approach: | They propose a hierarchical graph-based agent that represents complexity and a progressive strategy for step-by-step verification, refinement, and consistent context management. |
| Outcome: | The proposed agent surpasses state-of-the-art baselines on the MATH dataset and performs better on InfiAgent-DABench. |
Bridging the Pose-Semantic Gap: A Cascade Framework for Text-Based Person Anomaly Search (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for person anomaly search fail to address the complexities of real-world security, authors say . Existing approaches fail to detect subtle semantic distinctions, authors argue . |
| Approach: | They propose a framework that decouples retrieval into two stages . structure-aware coarse retrieval and detective squad interaction are proposed . |
| Outcome: | The proposed framework achieves state-of-the-art performance by balancing efficiency and semantic reasoning. |
Tell-and-Answer: Towards Explainable Visual Question Answering using Attributes and Captions (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to visual question answering represent images using pre-trained CNNs . but they rarely provide any insight, apart from the answer, into the VQA process . |
| Approach: | They propose to break up the end-to-end VQA into two steps: explaining and reasoning . they first extract attributes and generate descriptions as explanations for an image . a reasoning module utilizes these explanations in place of the image to infer an answer . |
| Outcome: | The proposed system achieves comparable performance with baselines, but with added benefits of explanability and the ability to improve with higher quality explanations. |
Structured Pruning for Efficient Generative Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Large-scale generative Pre-trained Language Models (PLMs) are limited in their deployment in real-world applications. |
| Approach: | They propose to prune the feed-forward networks of generative pre-trained language models to smaller widths without designing extra operators. |
| Outcome: | The proposed method achieves 1.51x/6.96x inference speedup on GPU/CPU with 67% size reduction. |
Compression of Generative Pre-trained Language Models via Quantization (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to compress generative pre-trained language models fail on generative tasks due to homogeneous word embeddings and limited memory. |
| Approach: | They propose a token-level contrastive distillation method to learn distinguishable word embeddings and a module-wise dynamic scaling method to make quantizers adaptive to different modules. |
| Outcome: | The proposed method outperforms the state-of-the-art compression methods on generative PLMs by a clear margin. |
ProductResearch: Training E-Commerce Deep Research Agents via Multi-Agent Synthetic Trajectory Distillation (2026.acl-industry)
Copied to clipboard
| Challenge: | Existing LLM-based agents lack the interaction depth and contextual breadth required for complex product research. |
| Approach: | They propose a multi-agent framework that synthesizes high-fidelity tool-use trajectories for training robust e-commerce shopping agents. |
| Outcome: | The proposed framework synthesizes high-fidelity, long-horizon tool-use trajectories for training robust e-commerce shopping agents. |
Backdooring Neural Code Search (2023.acl-long)
Copied to clipboard
| Challenge: | Neural code search models are used to find code snippets from online repositories . however, their security aspect is rarely studied . |
| Approach: | They propose to use off-the-shelf code snippets from online repositories to find desired code . they propose to inject a backdoor into neural code search models which return buggy code if attacker modifies one variable/function name . |
| Outcome: | The proposed attack outperforms baselines on two neural code search models by 60%. |
MARS-Bench: A Multi-turn Athletic Real-world Scenario Benchmark for Dialogue Evaluation (2025.findings-emnlp)
Copied to clipboard
Chenghao Yang, Yinbo Luo, Zhoufutu Wen, Qi Chu, Tao Gong, Longxiang Liu, Kaiyuan Zhang, Jianpeng Jiao, Ge Zhang, Wenhao Huang, Nenghai Yu
| Challenge: | Large Language Models (LLMs) have been widely adopted in real-world dialogue applications, but their robustness is criticized all along. |
| Approach: | They propose to use play-by-play text commentary to build a multi-turn athletic real-world scenario dialogue benchmark to evaluate three critical aspects of multi-turned conversations: ultra multi- turn, interactive multi-twist, and cross-turn tasks. |
| Outcome: | The proposed benchmarks outperform open-source LLMs on three critical aspects of multi-turn conversations: ultra multi-turned, interactive multi- turn, and cross-turn tasks. |
CReSE: Benchmark Data and Automatic Evaluation Framework for Recommending Eligibility Criteria from Clinical Trial Information (2024.findings-eacl)
Copied to clipboard
| Challenge: | Eligibility criteria (EC) are defined as a set of conditions an individual must meet to participate in a clinical trial. |
| Approach: | They propose to recommend EC based on clinical trial information, including trial titles, and introduce an automatic evaluation framework to assess clinical validity of the EC recommendation model. |
| Outcome: | The proposed model outperforms existing language models pre-trained on the biomedical domain in EC clustering. |
M-Ped: Multi-Prompt Ensemble Decoding for Large Language Models (2025.findings-emnlp)
Copied to clipboard
Jiaxin Guo, Daimeng Wei, Yuanchang Luo, Hengchao Shang, Zongyao Li, Jinlong Yang, Zhanglin Wu, Zhiqiang Rao, Shimin Tao, Hao Yang
| Challenge: | a new ensemble decoding approach enhances the performance of Large Language Models. |
| Approach: | They propose a multi-prompt ensemble decoding approach to enhance LLM performance . they submit n variations of prompts with X to LLMs in batch mode to decode and derive probability distributions . |
| Outcome: | The proposed method improves pass@k rates, LENS metrics and BLEU scores on diverse NLP tasks. |
TemplateRL: Structured Template-Guided Reinforcement Learning for LLM Reasoning (2026.findings-acl)
Copied to clipboard
Jinyang Wu, Chonghua Liao, Mingkuan Feng, Shuai Zhang, Zhengqi Wen, Haoran Luo, Ling Yang, Huazhe Xu, Jianhua Tao
| Challenge: | Existing RL methods rely on unstructured self-sampling to fit scalar rewards, resulting in inefficient rollouts. |
| Approach: | They propose a structured template-guided RL framework that augments policy optimization with explicit template guidance. |
| Outcome: | Experiments show that TemplateRL outperforms GRPO and GRPI by 99% on AIME and 41% on AMC with superior stability on weak models and remarkable cross-domain generalization. |
AEQ-Bench: Measuring Empathy of Omni-Modal Large Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks focus on cognitive abilities, such as knowledge retrieval, complex reasoning, and instruction following, largely overlooking empathy evaluation. |
| Approach: | They propose to benchmark two core empathetic capabilities of omnimodal large models (OLMs) generating empatries by comprehending affective cues from multi-modal inputs and judging empathy of audio responses without relying on text transcription. |
| Outcome: | The proposed benchmark outperforms existing models with audio output capabilities but is unreliable for evaluating fine-grained paralinguistic expressiveness. |
When Rules Learn: A Self-Evolving Agent for Legal Case Retrieval (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing dense retrieval methods have achieved notable progress, but their effectiveness in legal case retrieval remains limited. |
| Approach: | They propose a self-evolving framework for rule-driven query rewriting that enhances BM25 without any parameter training. |
| Outcome: | The proposed framework outperforms non-evolutionary baselines, including human-designed rules and greedy rule selection, especially when powered by a high-capacity core LLM. |
Learn to Memorize: Scalable Continual Learning in Semiparametric Models with Mixture-of-Neighbors Induction Memory (2025.acl-long)
Copied to clipboard
| Challenge: | Semiparametric language models (LMs) use static storage, which lacks learning capability and is disconnected from the internal information flow of the parametric models. |
| Approach: | They reconceptualize the non-parametric memory represented by kNN-LM as a learnable Mixture-of-Neighbors Induction Memory (MoNIM) this synergizes the induction capabilities of attention heads with the memorization strength of feed-forward networks . |
| Outcome: | The proposed model is a learnable Mixture-of-neighbors induction memory (MoNIM) it synergizes the induction capabilities of attention heads with the memorization strength of feed-forward networks (FFNs). |
O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning (2026.findings-acl)
Copied to clipboard
Haotian Luo, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Xiaochun Cao, Dacheng Tao, Naiqiang Tan, Li Shen
| Challenge: | Recent long-thought reasoning models adopt extended reasoning processes similar to how humans ponder over complex problems. |
| Approach: | They propose a model that uses RL-style fine-tuning to reduce inference overhead while maintaining accuracy. |
| Outcome: | The proposed model reduces inference overhead while maintaining accuracy. |
Beyond Examples: Towards Automated Thought-level In-Context Reasoning for Large Language Models (2026.acl-long)
Copied to clipboard
Jinyang Wu, Mingkuan Feng, Shuai Zhang, Feihu Che, Zhengqi Wen, Chonghua Liao, Ling Yang, Haoran Luo, Zheng Lian, Jianhua Tao
| Challenge: | In-context learning (ICL) struggles with complex reasoning due to superficial, example-level implicit imitation. |
| Approach: | They propose an automated method that shifts from surface-level examples to more guidance-oriented thought patterns. |
| Outcome: | The proposed method achieves 80.6% accuracy on MATH and 62.5% on AMC, surpassing GPT-4o’s 77.2% and 57.5% accuracy. |
RENOVI: A Benchmark Towards Remediating Norm Violations in Socio-Cultural Conversations (2024.findings-naacl)
Copied to clipboard
Haolan Zhan, Zhuang Li, Xiaoxi Kang, Tao Feng, Yuncheng Hua, Lizhen Qu, Yi Ying, Mei Rianto Chandra, Kelly Rosalin, Jureynolds Jureynolds, Suraj Sharma, Shilin Qu, Linhao Luo, Ingrid Zukerman, Lay-Ki Soon, Zhaleh Semnani Azad, Reza Haf
| Challenge: | Norm violations occur when individuals fail to conform to culturally accepted behaviors, which may lead to potential conflicts. |
| Approach: | They propose to use a large corpus of 9,258 multi-turn dialogues annotated with social norms to equip AI systems with a remediation ability. |
| Outcome: | The proposed system can understand and remediate norm violations step by step. |
ChatASU: Evoking LLM’s Reflexion to Truly Understand Aspect Sentiment in Dialogues (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies on interactive ASU ignore the coreference issue for opinion targets while this phenomenon is ubiquitous in interactive scenarios especially dialogues, limiting the ASU performance. |
| Approach: | They propose a Chat-based Aspect Sentiment Understanding task that integrates various NLP tasks with the chat paradigm and propose 'trusted self-reflexion' approach with ChatGLM as backbone to address aspect coreference issue. |
| Outcome: | The proposed task outperforms state-of-the-art baselines and shows that it is highly effective. |
Enhancing LLM-based Search Agents via Contribution Weighted Group Relative Policy Optimization (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to training large language models suffer from unstable value estimation, whereas outcome supervision struggles with credit assignment due to sparse, trajectory-level rewards. |
| Approach: | They propose a framework that integrates process supervision into group relative policy optimization. |
| Outcome: | The proposed framework outperforms standard GRPO on knowledge-intensive benchmarks by 5.0% and 6.3% on Qwen3-1.7B. |
LEAF: Large Language Diffusion Model for Time Series Forecasting (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work has applied large language models (LLMs) into time series forecasting, but they lack an understanding of holistic temporal patterns with potential error accumulation. |
| Approach: | They propose a framework that marries Larg e Langu age Diffusion Model with time series forecasting (LEAF) they propose converting time series into tokens and adopting language diffusion models to capture temporal dependencies. |
| Outcome: | The proposed framework generates future predictions with a diffusion model from a holistic view. |
CFBench: A Comprehensive Constraints-Following Benchmark for LLMs (2025.acl-long)
Copied to clipboard
Tao Zhang, ChengLIn Zhu, Yanjun Shen, Wenjing Luo, Yan Zhang, Hao Liang, Tao Zhang, Fan Yang, Mingan Lin, Yujing Qiao, Weipeng Chen, Bin Cui, Wentao Zhang, Zenan Zhou
| Challenge: | Existing evaluations of Large Language Models (LLMs) focus on fragmented constraints or narrow scenarios, but they overlook the comprehensiveness and authenticity of constraints from the user’s perspective. |
| Approach: | They propose a Chinese Comprehensive Constraints Following Benchmark for LLMs that compiles constraints from real-world instructions and constructs a systematic framework for constraint types. |
| Outcome: | The proposed framework integrates multi-dimensional assessment criteria with requirement prioritization, covering various perspectives of constraints, instructions, and requirement fulfillment. |
Playing 20 Question Game with Policy-Based Reinforcement Learning (D18-1)
Copied to clipboard
| Challenge: | The 20 Questions (Q20) game encourages deductive reasoning and creativity. |
| Approach: | They propose a policy-based Reinforcement Learning method which learns optimal question selection . the method is robust to noisy answers and uses a reward network to estimate the more informative reward . |
| Outcome: | The proposed method outperforms an entropy-based engineering system and has competitive performance in noisy-free simulation environment. |
ELBA-Bench: An Efficient Learning Backdoor Attacks Benchmark for Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing backdoor models are limited in coverage of attack, system integrity and backdoor alignment . ELBA-Bench provides over 1300 experiments encompassing 12 attack methods, 18 datasets, and 12 LLMs. |
| Approach: | They propose a framework that allows attackers to inject backdoor through parameter efficient fine-tuning or without fine-uning techniques. |
| Outcome: | ELBA-Bench provides over 1300 experiments encompassing 12 attack methods, 18 datasets, and 12 LLMs. |
Mitigating Coordinate Prediction Bias from Positional Encoding Failures (2026.findings-acl)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) excel at general vision-language tasks, but precise coordinate prediction remains a challenge. |
| Approach: | They propose a training-free, inference-time correction method to correct VPEs . they isolate position-unconditioned tendencies by shuffling VPE and use it to steer digit decoding . |
| Outcome: | The proposed method is training-free, inference-time correction method . it effectively rectifies coordinate drift, yielding consistent improvements without retraining . |
Polymorphic Universal Transformer (2026.acl-long)
Copied to clipboard
Yilong Chen, Zitian Gao, Yihao Xiao, Jason Klein Liu, Xinyu Yang, Yifan Luo, Haoming Luo, Zhengmao Ye, Tingwen Liu, Ran Tao, Bryan Dai
| Challenge: | Compute Distribution Skew is a pathological phenomenon in ultra-deep recurrent models . it causes over-smoothing, representation rank collapse, and degraded reasoning performance. |
| Approach: | They propose a dynamic architecture that redefines recursive computation by decoupling parameter count from depth. |
| Outcome: | The proposed model significantly improves representation rank and reasoning robustness while reducing computation by 64.7%. |
EHRAG: Bridging Semantic Gaps in Lightweight GraphRAG via Hybrid Hypergraph Construction and Retrieval (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing lightweight approaches to retrieval-augmented generation fail to capture latent semantic connections between disjoint entities. |
| Approach: | They propose a lightweight RAG framework that constructs a hypergraph capturing both structure and semantic relationships using a hybrid structural-semantic retrieval mechanism. |
| Outcome: | EHRAG outperforms state-of-the-art methods on four datasets while maintaining zero token consumption. |
Breaking the Impasse: Dual-Scale Evolutionary Policy Training for Social Language Agents (2026.acl-long)
Copied to clipboard
| Challenge: | Reinforcement Learning with Verifiable Rewards (RLVR) is effective for closed-ended tasks, but it is not applicable to open-ended social language games. |
| Approach: | They propose a method that uses a time-scaled evolutionary perception mechanism to detect impasse by quantifying dual-scale value baseline divergence alongside match entropy. |
| Outcome: | Experiments on multiple social language games show that the proposed method outperforms baselines and avoids policy degeneration. |
One2Set: Generating Diverse Keyphrases as a Set (2021.acl-long)
Copied to clipboard
| Challenge: | Recent keyphrase generation models are wrongly imposing a predefined order on keyphrases . a new training paradigm is proposed to concatenate keyphrase sequences in parallel . |
| Approach: | They propose a training paradigm that concatenates keyphrases in a predefined order . they propose combining a fixed set of learned control codes with a bipartite matching mechanism . |
| Outcome: | The proposed model outperforms the state-of-the-art methods on multiple benchmarks. |