Papers by Zheng Du
ToolGrad: Efficient Tool-use Dataset Generation with Textual “Gradients” (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior work synthesizes tool-use LLM datasets by first generating a user query, then complex tool-using annotations like DFS. |
| Approach: | They propose an agentic framework that synthesizes user queries and generates valid tool-use chains . they propose a dataset with more complex tool use, lower cost, and almost 100% pass rate . |
| Outcome: | Experiments show that tools trained on ToolGrad outperform expensive baseline datasets and proprietary LLMs. |
ReCo: Reliable Causal Chain Reasoning via Structural Causal Recurrent Neural Networks (2022.emnlp-main)
Copied to clipboard
| Challenge: | Causal chain reasoning models suffer from two main transitive problems: threshold effect and scene drift. |
| Approach: | They propose a framework that uses exogenous variables to represent causal pairs and estimates the threshold and scene contradictions using structural causal recurrent neural networks. |
| Outcome: | The proposed framework outperforms baselines on Chinese and English CCR datasets. |
Model Composition for Multimodal Large Language Models (2024.acl-long)
Copied to clipboard
Chi Chen, Yiyang Du, Zheng Fang, Ziyue Wang, Fuwen Luo, Peng Li, Ming Yan, Ji Zhang, Fei Huang, Maosong Sun, Yang Liu
| Challenge: | Existing methods for creating versatile MLLMs rely on joint training with paired instruction data, which is resource-intensive and challenging to extend to new modalities. |
| Approach: | They propose a new paradigm for multimodal large language models by reusing modality encoders and merging LLM parameters. |
| Outcome: | The proposed model retains the modal understanding capabilities of each original model. |
An Information Bottleneck Perspective for Effective Noise Filtering on Retrieval-Augmented Generation (2024.acl-long)
Copied to clipboard
Kun Zhu, Xiaocheng Feng, Xiyuan Du, Yuxuan Gu, Weijiang Yu, Haotian Wang, Qianglong Chen, Zheng Chu, Jingchang Chen, Bing Qin
| Challenge: | Experimental results show that retrieval-augmented generation improves accuracy and relevance of large language models. |
| Approach: | They propose to introduce the information bottleneck theory into retrieval-augmented generation by maximizing mutual information between compression and ground output while minimizing mutual information . |
| Outcome: | The proposed approach improves accuracy and correctness of answer generation and conciseness with 2.5% compression rate. |
LightThinker: Thinking Step-by-Step Compression (2025.emnlp-main)
Copied to clipboard
Jintian Zhang, Yuqi Zhu, Mengshu Sun, Yujie Luo, Shuofei Qiao, Lun Du, Da Zheng, Huajun Chen, Ningyu Zhang
| Challenge: | Recent advances in Large Language Models have demonstrated their remarkable capabilities in complex reasoning tasks, but their efficiency is hindered by the substantial memory and computational costs associated with generating lengthy tokens. |
| Approach: | They propose a method that dynamically compresses verbose thought steps into compact representations and discards original reasoning chains. |
| Outcome: | The proposed method reduces peak memory usage and inference time while maintaining competitive accuracy. |
What Makes AI Research Replicable? Executable Knowledge Graphs as Scientific Knowledge Representations (2026.acl-short)
Copied to clipboard
Yujie Luo, Zhuoyun Yu, Xuehai Wang, Yuqi Zhu, Ningyu Zhang, Lanning Wei, Lun Du, Da Zheng, Huajun Chen
| Challenge: | Existing approaches to replicate AI research are limited by insufficient background knowledge and the limitations of retrieval-augmented generation methods. |
| Approach: | They propose a pluggable, paper-centric knowledge base that integrates code snippets and technical insights extracted from scientific literature into a verifiable, executable representation. |
| Outcome: | The proposed knowledge base shows significant performance gains on paperBench when integrated into three agent frameworks with two different LLMs. |
GIFT: Guided Fine-Tuning and Transfer for Enhancing Instruction-Tuned Language Models (2026.acl-long)
Copied to clipboard
Zhiwen Ruan, Yichao Du, Jianjie Zheng, Longyue Wang, Yun Chen, Peng Li, Jinsong Su, Yang Liu, Guanhua Chen
| Challenge: | Existing adapter-based transfer methods treat instruction-tuned models as passive targets . direct fine-tuning can disrupt this delicate balance and lead to instability or performance degradation. |
| Approach: | They propose a framework that incorporates instruction-level guidance into task adaptation. |
| Outcome: | The proposed framework outperforms direct fine-tuning and representative transfer-based baselines while maintaining robust generalization and favorable test-time scaling behavior. |
KARPA: A Training-free Method of Adapting Knowledge Graph as References for Large Language Model’s Reasoning Path Aggregation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for large language models (LLMs) are limited by step-by-step decision-making on KGs, or require fine-tuning or pre-training on specific KG. |
| Approach: | They propose a framework that harnesses the global planning abilities of large language models (LLMs) for efficient and accurate KG reasoning. |
| Outcome: | Extensive experiments show that the proposed framework achieves state-of-the-art performance in KGQA tasks, delivering both high efficiency and accuracy. |
Spiral of Silence in Large Language Model Agents (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing theories of Spiral of Silence do not apply to large language models . |
| Approach: | They propose an evaluation framework for examining SoS in large language models . they consider four controlled conditions that vary the availability of "History" and "Persona" signals . |
| Outcome: | The proposed framework examines the SoS-like dynamics in large language models . it shows that history and persona together produce strong majority dominance . |
Large Language Models for Automated Open-domain Scientific Hypotheses Discovery (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing research on hypothetical induction is limited by the observation annotations in the dataset and the ground truth hypotheses are mostly commonsense knowledge. |
| Approach: | They propose a first dataset for social science academic hypotheses discovery using raw web corpus as observations and propose valid, useful scientific hypothese . they propose 'a multi-module framework' that includes feedback mechanisms to boost performance. |
| Outcome: | The proposed dataset generates valid, novel, and helpful scientific hypotheses, even new to humanity, using open-domain data and a web corpus as observations. |
Bridging Distance and Spectral Positional Encodings via Anchor-Based Diffusion Geometry Approximation (2026.acl-long)
Copied to clipboard
| Challenge: | Molecular graph learning benefits from positional signals that capture local neighborhoods and global topology. |
| Approach: | They propose to use anchor-based distance encodings to approximate diffusion geometry. |
| Outcome: | The proposed model outperforms models without positional encodings on DrugBank with a shared GNP-based DDI backbone. |
Can We Predict Before Executing Machine Learning Agents? (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to scientific discovery rely on expensive physical execution . a Generate-Execute-Feedback paradigm is costly and slow . |
| Approach: | They propose to internalize execution priors to substitute costly runtime checks with instantaneous predictive reasoning, drawing inspiration from World Models. |
| Outcome: | The proposed framework achieves 61.5% accuracy and robust confidence calibration when primed with a Verified Data Analysis Report. |
AdaV: Adaptive Text-visual Redirection for Vision-Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Vision-language models often generate excessive visual tokens, leading to poor performance . a novel training-free visual token pruning method is proposed to improve performance despite the computational cost associated with VLMs. |
| Approach: | They propose a training-free visual token pruning method that reduces biased token pruning . they plan to open-source the code upon publication . |
| Outcome: | The proposed method reduces biased token pruning and enhances model robustness with limited visual token budget. |
Investigating Multi-Hop Factual Shortcuts in Knowledge Editing of Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Recent work has demonstrated the power of large language models in recalling knowledge and reasoning. |
| Approach: | They propose to erase shortcut neurons to mitigate the associated risks . 20% of the failures are attributed to shortcuts, they find . |
| Outcome: | The proposed approach reduces failures in multi-hop knowledge editing caused by shortcuts by 20% . |
DualRAG: A Dual-Process Approach to Integrate Reasoning and Retrieval for Multi-Hop Question Answering (2025.acl-long)
Copied to clipboard
Rong Cheng, Jinyi Liu, Yan Zheng, Fei Ni, Jiazhen Du, Hangyu Mao, Fuzheng Zhang, Bo Wang, Jianye Hao
| Challenge: | Existing approaches to multi-hop question answering struggle to identify and organize dynamic knowledge . et al., 2023; Liu e.t. al. 2023) suggest a dual-process framework for multi-step reasoning . |
| Approach: | They propose a synergistic dual-process framework that integrates reasoning and retrieval. |
| Outcome: | The proposed framework improves answer accuracy and coherence even in smaller-scale models. |
OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversation (2025.acl-long)
Copied to clipboard
Qinglin Zhang, Luyao Cheng, Chong Deng, Qian Chen, Wen Wang, Siqi Zheng, Jiaqing Liu, Hai Yu, Chao-Hong Tan, Zhihao Du, ShiLiang Zhang
| Challenge: | Full-duplex spoken dialogue systems allow simultaneous bidirectional communication . low latency and natural interactions in full-duplice systems remains a challenge . |
| Approach: | They propose a multi-stage post-training scheme that adapts a text large language model into a speech-text dialogue LLM. |
| Outcome: | The proposed model can model human conversation behaviors with low latency and natural interactions with low delay. |
Retrieval-Augmented Language Models are Mimetic Theorem Provers (2025.findings-emnlp)
Copied to clipboard
Wenjie Yang, Ruiyuan Huang, Jiaxing Guo, Zicheng Lyu, Tongshan Xu, Shengzhong Zhang, Lun Du, Da Zheng, Zengfeng Huang
| Challenge: | Large language models often fail to provide rigorous proof-based reasoning for research-level mathematics. |
| Approach: | They propose a simple yet effective RAG framework that augments retrieved proofs with queries and document contexts to improve retrieval performance. |
| Outcome: | The proposed framework improves retrieval performance by 34.19% . dual RAG can be used to prove research-level theorems in theoretical machine learning . |
Learning to Evolve: A Self-Improving Framework for Multi-Agent Systems via Textual Parameter Graph Optimization (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for designing and optimizing multi-agent systems are static and do not learn from experience. |
| Approach: | They propose a framework that enables a multi-agent system to learn to evolve . they use "textual gradients" to pinpoint failures and suggest granular modifications . |
| Outcome: | a new framework enables a multi-agent system to learn to evolve . it learns from historical optimization experiences to improve its performance . |
SLIM: Let LLM Learn More and Forget Less with Soft LoRA and Identity Mixture (2025.naacl-long)
Copied to clipboard
| Challenge: | balancing the training budget, downstream performance, and general capabilities of large language models remains a challenge in many applications. |
| Approach: | They propose a mixture of expert framework based on Soft LoRA and Identity Mixture . SLIM allows dynamic routing between LoRA adapters and identity layers . |
| Outcome: | The proposed framework reduces training cost while maintaining general capabilities . it can be open-sourced upon publication. |
A User-Centered, Interactive, Human-in-the-Loop Topic Modelling System (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent research has demonstrated the value of user feedback, but there are still issues to consider, such as the difficulty in tracking changes and comparing different models. |
| Approach: | They propose a human-in-the-loop topic modeling system that integrates users’ knowledge into the modelling process, enabling them to refine the model iteratively. |
| Outcome: | The proposed system is based on a series of user studies to assess its performance in progressively more realistic applications. |
WebOlympus: An Open Platform for Web Agents on Live Websites (2024.emnlp-demo)
Copied to clipboard
| Challenge: | Web agents are emerging as powerful tools for automating tasks in cyberspace . however, there is a lack of standardized and user-friendly tools for research and development . |
| Approach: | They propose an open platform for web agents operating on live websites with a Chrome extension and a safety monitor module to ensure their trustworthiness. |
| Outcome: | WebOlympus is an open platform for web agents operating on live websites. |
GrandGuard: Taxonomy, Benchmark, and Safeguards for Elderly-Chatbot Interaction Safety (2026.findings-acl)
Copied to clipboard
Changxuan Fan, Xi Yang, Yueyuan Zheng, Bin Zhou, Yuanping Wang, Wenbin Hu, Huihao Jing, Ki Sen Hung, Dazhao Du, Haoran Li, Janet Hui-wen Hsiao, Yangqiu Song
| Challenge: | a survey of older adults shows that many LLMs mishandle elderly-specific contextual risks. |
| Approach: | They propose a framework to assess elderly-specific contextual risks in LLM interactions . they use a taxonomy to identify 50 fine-grained risk types across mental well-being, financial, medical, toxicity, and privacy domains . |
| Outcome: | a new framework assesses elderly-specific contextual risks in LLM interactions . it achieves 96.2% and 90.9% unsafe-prompt detection accuracy, respectively . |
Enabling Agents to Communicate Entirely in Latent Space (2026.acl-long)
Copied to clipboard
Zhuoyun Du, Runze Wang, Huiyu Bai, Zouying Cao, Xiaoyong Zhu, Yu Cheng, Bo Zheng, Wei Chen, Haochao Ying
| Challenge: | Natural language is the de facto communication medium for LLM-based agents, but it presents a fundamental constraint . natural language downsampling limits the depth and nuance of information that can be transmitted . et al.: inter-agent latent space communication is a promising paradigm for solving complex tasks . |
| Approach: | They propose a paradigm that leverages the last hidden states of an LLM as a representation of its thought for direct communication. |
| Outcome: | The proposed paradigm outperforms fine-tuned chain-of-thought prompting and single-agent baselines even across heterogeneous models. |
COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for Alignment with Human Values (2026.findings-eacl)
Copied to clipboard
Siwei Wu, JinCheng Ren, Xeron Du, Shuyue Guo, Xingwei Qu, Yiming Liang, Jie Liu, Yunwen Li, Tyler Loakman, Tianyu Zheng, Boyu Feng, Huaqing Yuan, Zili Wang, Jiaheng Liu, Wenhao Huang, Chenglin Cai, Haoran Que, Jian Yang, Yuelin Bai, Zekun Moore Wang, Zhouliang Yu, Qunshu Lin, Ding Pan, Yuchen Eleanor Jiang, Tiannan Wang, Wangchunshu Zhou, Shenzhi Wang, Xingyuan Bu, Minghao Liu, Guoyin Wang, Ge Zhang, Chenghua Lin
| Challenge: | Existing Chinese preference datasets suffer from limited scale, restricted domain coverage, and insufficiently rigorous data validation. |
| Approach: | They propose an LLM-based data annotation pipeline with no human intervention to annotate Chinese preference datasets. |
| Outcome: | The proposed pipeline outperforms existing Chinese preference datasets on AlignBench and Chinese Reward Benchmark. |
Speaker Overlap-aware Neural Diarization for Multi-party Meeting Analysis (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to speaker diarization treat speaker dependency and overlaps as multi-label classification problems. |
| Approach: | They propose to reformulate overlapped speaker diarization task as a single-label prediction problem via power set encoding (PSE) to overcome the disadvantages, they propose a speaker overlap-aware neural diarisation model which incorporates a context-independent scorer and a contextual-dependent score. |
| Outcome: | The proposed model outperforms the state-of-the-art methods on speaker voice activity detection and improves relative diarization error reduction by 6.30%. |
Taming Language Models for Text-attributed Graph Learning with Decoupled Aggregation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to learning text-attributed graphs neglect interaction between textual and structural information. |
| Approach: | They propose a framework that integrates textual and structural information into TAG learning . they propose combining semantic aggregation and structural aggregations to improve learning a . |
| Outcome: | The proposed framework outperforms state-of-the-art learning methods while requiring less resources. |
Retrieval-Augmented Process Reward Model for Generalizable Mathematical Reasoning (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have advanced mathematical reasoning, but they still struggle with out-of-distribution (OOD) issues. |
| Approach: | They propose a framework to evaluate the logical validity of reasoning steps . they retrieves semantically similar questions and steps for PRM as a warmup . |
| Outcome: | The proposed framework outperforms baseline models on multiple real-world datasets. |
MiMoTable: A Multi-scale Spreadsheet Benchmark with Meta Operations for Table Reasoning (2025.coling-main)
Copied to clipboard
| Challenge: | Existing benchmarks for table reasoning are incomplete due to the complexity of the tables and user questions in real-world applications. |
| Approach: | They propose a Multi-scale spreadsheet benchmark with Meta operations for Table reasoning that incorporates two key features and a new criterion with six categories of meta operations for measuring the difficulty of each question. |
| Outcome: | The proposed model outperforms Claude-3.5-Sonnet with 77.4% accuracy on the existing benchmarks. |
NiuTrans.LMT: Toward Inclusive and Scalable Multilingual Machine Translation with LLMs (2026.acl-long)
Copied to clipboard
Yingfeng Luo, Ziqiang Xu, Yuxuan Ouyang, MuRun Yang, DingYang Lin, Kaiyan Chang, Tong Zheng, Bei Li, Peinan Feng, Quan Du, Tong Xiao, JingBo Zhu
| Challenge: | Large language models have significantly advanced Multilingual Machine Translation (MMT) yet scaling to many languages while maintaining robust performance across directions remains challenging. |
| Approach: | They propose a strategy to reduce the number of translations in one direction . they propose auxiliary parallel sentences to promote cross-lingual transfer . |
| Outcome: | The proposed model performs on par with or better than substantially larger baselines. |
Understanding Gender Bias in Knowledge Base Embeddings (2022.acl-long)
Copied to clipboard
| Challenge: | Knowledge base (KB) embeddings have been shown to contain gender biases . authors develop two new bias measures to quantify them and trace their origins in KB . |
| Approach: | They propose two ways to quantify gender biases in knowledge base (KB) embeddings . they use the influence function to inspect the contribution of each triple in KB to the overall group bias . |
| Outcome: | The proposed measures are compared with real-world census data to examine gender biases. |
COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning (2025.findings-naacl)
Copied to clipboard
Yuelin Bai, Xeron Du, Yiming Liang, Leo Jin, Junting Zhou, Ziqiang Liu, Feiteng Fang, Mingshan Chang, Tianyu Zheng, Xincheng Zhang, Nuo Ma, Zekun Moore Wang, Ruibin Yuan, Haihong Wu, Hongquan Lin, Wenhao Huang, Jiajun Zhang, Chenghua Lin, Jie Fu, Min Yang, Shiwen Ni, Ge Zhang
| Challenge: | Existing datasets for Chinese instruction tuning are not well-aligned with Chinese users’ interaction patterns. |
| Approach: | They propose to use Chinese instruction tuning datasets to improve instruction fine-tuning for Chinese users. |
| Outcome: | The proposed dataset shows that Chinese models achieve competitive performance in diverse benchmarks. |
BeamAggR: Beam Aggregation Reasoning over Multi-source Knowledge for Multi-hop Question Answering (2024.acl-long)
Copied to clipboard
Zheng Chu, Jingchang Chen, Qianglong Chen, Haotian Wang, Kun Zhu, Xiyuan Du, Weijiang Yu, Ming Liu, Bing Qin
| Challenge: | Large language models (LLMs) have demonstrated strong reasoning capabilities, but they still suffer from factual errors when tackling knowledge-intensive tasks. |
| Approach: | They propose a reasoning framework for knowledge-intensive multi-hop QA that prioritizes promising answers at each hop of question. |
| Outcome: | The proposed framework outperforms SOTA methods on four open-domain multi-hop reasoning datasets by 8.5%. |
UNIVID: Unified Vision-Language Model for Video Moderation (2026.acl-industry)
Copied to clipboard
Kejuan Yang, Yizhuo Zhang, Mingyuan Du, Yue Zhang, Dixin Zheng, Kaili Zhao, Yang Xiao, Hanzhong Liang, Kenan Xiao
| Challenge: | Existing video moderation systems rely on fragmented black-box classification models that are difficult to maintain and lack transparency. |
| Approach: | They propose a Unified Vision-Language model for Video Moderation that generates policy-aware captions that serve as an interpretable intermediate representation. |
| Outcome: | The proposed model reduces violation leakage and overkill rate by 42.7% while reducing maintenance costs. |
VC4VG: Optimizing Video Captions for Text-to-Video Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in text-to-video generation highlight the critical role of high-quality video-text pairs in training models capable of producing coherent and instruction-aligned videos. |
| Approach: | They propose a caption optimization framework tailored to the needs of T2V models. |
| Outcome: | The proposed framework improves video caption quality and video generation performance. |
T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Step (2024.acl-long)
Copied to clipboard
Zehui Chen, Weihua Du, Wenwei Zhang, Kuikun Liu, Jiangning Liu, Miao Zheng, Jingming Zhuo, Songyang Zhang, Dahua Lin, Kai Chen, Feng Zhao
| Challenge: | Existing studies evaluate the tool utilization ability of large language models based on the final output or only consider the single-step tool calling. |
| Approach: | They propose a new approach to evaluate the tool utilization capability of large language models (LLMs) they decompose the tool usage into multiple sub-processes, including instruction following, planning, reasoning, retrieval, understanding, and review. |
| Outcome: | The proposed model exhibits consistency with the outcome-oriented evaluation and provides a more fine-grained analysis of the capabilities of LLMs. |