Papers by Weiwei Liu
Too Long, Do Re-weighting for Efficient LLM Reasoning Compression (2026.acl-long)
Copied to clipboard
Zhong-Zhi Li, Xiao Liang, Zihao Tang, Lei Ji, Peijie Wang, Haotian Xu, Xing W, Haizhen Huang, Weiwei Deng, Yeyun Gong, Zhijiang Guo, Xiao Liu, Fei Yin, Cheng-Lin Liu
| Challenge: | Large Language Models (LLMs) have recently achieved remarkable progress on complex reasoning tasks by leveraging extended Chain-of-Thought (CoT) techniques. |
| Approach: | They propose a method that uses Extended Chain-of-Thought (EFT) to reduce the number of output tokens by nearly 40% while maintaining the accuracy of the reasoning. |
| Outcome: | The proposed method reduces the number of output tokens by nearly 40% while maintaining the accuracy of the reasoning. |
Tchebycheff Procedure for Multi-task Text Classification (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for text classification assume that multitask text classification problems are convex multiobjective optimization problems. |
| Approach: | They propose a Tchebycheff procedure to optimize multi-task classification problems without convex assumption. |
| Outcome: | The proposed method is able to find an arbitrary Pareto optimal solution in the PareTO set if the problem is convex, but excludes many Paret optimal solutions from its search scope. |
Context-DPO: Aligning Language Models for Context-Faithfulness (2025.findings-acl)
Copied to clipboard
Baolong Bi, Shaohan Huang, Yiwei Wang, Tianchi Yang, Zihan Zhang, Haizhen Huang, Lingrui Mei, Junfeng Fang, Zehao Li, Furu Wei, Weiwei Deng, Feng Sun, Qi Zhang, Shenghua Liu
| Challenge: | Context-DPO is the first alignment method specifically designed to enhance contextfaithfulness for large language models. |
| Approach: | They propose a benchmark that simulates Retrieval-Augmented Generation scenarios with knowledge conflicts to evaluate context-faithfulness. |
| Outcome: | The proposed method improves LLMs' context-faithfulness by 35% to 280% over open-source models. |
MAIN: Mutual Alignment Is Necessary for instruction tuning (2025.emnlp-main)
Copied to clipboard
Fanyi Yang, Jianfeng Liu, Xin Zhang, Haoyu Liu, Xixin Cao, Yuefeng Zhan, Hao Sun, Weiwei Deng, Feng Sun, Qi Zhang
| Challenge: | Instruction tuning has enabled large language models to achieve remarkable performance, yet its success heavily depends on the availability of high-quality instruction-response pairs. |
| Approach: | They propose a mutual alignment framework which enforces coherence between instructions and responses through mutual constraints. |
| Outcome: | The proposed framework generalizes well across model architectures and sizes, achieving state-of-the-art performance on LLaMA, Mistral, and Qwen models across diverse benchmarks. |
FineCops-Ref: A new Dataset and Task for Fine-Grained Compositional Referring Expression Comprehension (2024.emnlp-main)
Copied to clipboard
| Challenge: | Referring Expression Comprehension (REC) is a cross-modal task that objectively evaluates the capabilities of language understanding, image comprehension, and language-to-image grounding. |
| Approach: | They propose to use a new reference expression comprehension (REC) dataset to evaluate the capabilities of language understanding, image comprehension, and language-to-image grounding. |
| Outcome: | The proposed model is able to reject scenarios where the target object is not visible in the image, a key aspect often overlooked in existing models and approaches. |
HD-Eval: Aligning Large Language Model Evaluators Through Hierarchical Criteria Decomposition (2024.acl-long)
Copied to clipboard
Yuxuan Liu, Tianchi Yang, Shaohan Huang, Zihan Zhang, Haizhen Huang, Furu Wei, Weiwei Deng, Feng Sun, Qi Zhang
| Challenge: | Large language models (LLMs) are a promising alternative to expensive human evaluations. |
| Approach: | They propose a framework that iteratively aligns LLM-based evaluators with human preference . they decompose a given evaluation task into finer-grained criteria . |
| Outcome: | The proposed framework iteratively aligns LLM-based evaluators with human preference . it decomposes a given evaluation task into finer-grained criteria . the framework is efficient to train and more explainable than relying solely on prompts . |
Mixture-of-Minds: Multi-Agent Reinforcement Learning for Table Understanding (2026.acl-long)
Copied to clipboard
Yuhang Zhou, Mingrui Zhang, Ke Li, Mingyi Wang, Qiao Liu, Qifei Wang, Jiayi Liu, Fei Liu, Serena Li, Weiwei LI, Mingze Gao, Abhishek Kumar, Xiangjun Fan, Zhuokai Zhao, Lizhu Zhang
| Challenge: | Large language models (LLMs) have shown promise on understanding and reasoning over tables, but current approaches remain limited. |
| Approach: | They propose a multi-agent framework that decomposes table reasoning into three specialized roles: planning, coding, and answering. |
| Outcome: | The proposed framework decomposes table reasoning into three specialized roles: planning, coding, and answering. |
Se2: Sequential Example Selection for In-Context Learning (2024.findings-acl)
Copied to clipboard
| Challenge: | Prior work has explored the selection of examples for in-context learning, neglecting the internal relationships between examples and exist an inconsistency between training and inference. |
| Approach: | They propose a sequential-aware method that leverages the LLM’s feedback on varying context, aiding in capturing inter-relationships and sequential information among examples. |
| Outcome: | Experiments on 23 NLP tasks show that Se2 surpasses baselines and achieves 42% relative improvement over random selection. |
FlowHN: Adaptive Token Routing for Efficient Parallel Hybrid Networks (2026.acl-industry)
Copied to clipboard
| Challenge: | Existing hybrids lack performance, latency, and cost-efficient scaling for production LLMs. |
| Approach: | They propose a deployment-oriented parallel hybrid architecture that enables deterministic conditional computation via FLOP-aware token circulation across attention and SSM branches. |
| Outcome: | FlowHN achieves 4 higher throughput and 15% higher MFU than current models while maintaining competitive accuracy on reasoning, coding, and long-context tasks. |
ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability (2026.acl-long)
Copied to clipboard
| Challenge: | Existing rerankers perform poorly in complex ranking scenarios due to the scarcity of reasoning-intensive training data. |
| Approach: | They propose an automated reasoning-intensive training framework which generates high-quality training labels from training queries and passages. |
| Outcome: | The proposed model outperforms baselines significantly and achieves much lower latency than the pointwise reranker. |
Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities (2025.naacl-long)
Copied to clipboard
Chung-En Sun, Xiaodong Liu, Weiwei Yang, Tsui-Wei Weng, Hao Cheng, Aidan San, Michel Galley, Jianfeng Gao
| Challenge: | Recent research shows that Large Language Models (LLMs) are vulnerable to automated jailbreak attacks. |
| Approach: | They propose a framework that crafts adversarial LLMs with enhanced jailbreak ability. |
| Outcome: | ADV-LLM significantly reduces the computational cost of generating adversarial suffixes while achieving nearly 100% ASR on various open-source LLMs. |
GeAR: Generation Augmented Retrieval (2025.findings-acl)
Copied to clipboard
Haoyu Liu, Shaohan Huang, Jianfeng Liu, Yuefeng Zhan, Hao Sun, Weiwei Deng, Feng Sun, Furu Wei, Qi Zhang
| Challenge: | Document retrieval techniques are used to compute semantic similarity between a query and documents, but the scalar similarity fails to reflect enough information, hindering the interpretation of retrieval results. |
| Approach: | They propose a method which improves the global document-query similarity through contrastive learning and integrates well-designed fusion and decoding modules. |
| Outcome: | The proposed method improves the global document-query similarity through contrastive learning and integrates well-designed fusion and decoding modules. |
Democratizing Reasoning Ability: Tailored Learning from Large Language Model (2023.emnlp-main)
Copied to clipboard
Zhaoyang Wang, Shaohan Huang, Yuxuan Liu, Jiahai Wang, Minghui Song, Zihan Zhang, Haizhen Huang, Furu Wei, Weiwei Deng, Feng Sun, Qi Zhang
| Challenge: | Large language models (LLMs) exhibit impressive emergent abilities in natural language processing, but their democratization is hindered due to huge computation requirements and closed-source nature. |
| Approach: | They propose a tailored learning approach to distill the exclusive reasoning ability to smaller LMs to facilitate democratization. |
| Outcome: | The proposed approach enables the democratization of the exclusive reasoning ability by leveraging the black-box model as a reasoning teacher. |
BanditMTL: Bandit-based Multi-task Learning for Text Classification (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to regularize task variance are unexplored in multi-task text classification. |
| Approach: | They propose a multi-task learning method based on adversarial multi-armed bandit to regularize the task variance by means of a mirror gradient ascent-descent algorithm. |
| Outcome: | The proposed method achieves state-of-the-art in multi-task text classification. |
FLOP-Efficient Training: Early Stopping Based on Test-Time Compute Awareness (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior work shows that increasing test-time compute (TTC) can improve accuracy of large language models. |
| Approach: | They propose a TTC-aware training algorithm that jointly selects a checkpoint and a corresponding TTC configuration to minimize training compute without sacrificing accuracy. |
| Outcome: | The proposed method reduces training compute by 92% while maintaining accuracy. |
Token-level Proximal Policy Optimization for Query Generation (2025.emnlp-main)
Copied to clipboard
Yichen Ouyang, Lu Wang, Fangkai Yang, Pu Zhao, Chenghua Huang, Jianfeng Liu, Bochen Pang, Yaming Yang, Yuefeng Zhan, Hao Sun, Qingwei Lin, Saravan Rajmohan, Weiwei Deng, Dongmei Zhang, Feng Sun
| Challenge: | Large Language Models (LLMs) have improved search engines and recommendation systems through their text understanding capabilities. |
| Approach: | They propose a token-level proximal policy optimization approach to empower LLMs to perform better in query generation through fine-tuning. |
| Outcome: | The proposed approach outperforms existing LLMs on an open-source and industrial dataset. |
How Large Language Models Encode Context Knowledge? A Layer-Wise Probing Study (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies have focused on enhancing the factualness of large language models using context knowledge. |
| Approach: | They propose to use ChatGPT to construct probing datasets that provide diverse and coherent evidence corresponding to various facts. |
| Outcome: | The proposed model can encode knowledge across different layers, and it is compared with existing models. |
Beyond the Context Window: Scaling Agentic RL via End-to-end Optimized Context Compression (2026.acl-long)
Copied to clipboard
| Challenge: | Existing reinforcement learning pipelines suffer from degraded instruction following, excessive rollout costs, and strict context limits. |
| Approach: | They propose a reinforcement learning (RL) fine-tuning of large language model (LLM) agents for long-horizon multi-turn tool use where context length quickly becomes a bottleneck. |
| Outcome: | The proposed framework improves the success rate while maintaining the same or even lower working context length compared to baselines. |
MGPO: Thinking with Images via Multi-Turn Grounding-Based Reinforcement Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | State-of-the-art large multimodal models face challenges when processing high-resolution images, as these inputs are converted into enormous visual tokens, many of which are irrelevant to the downstream task. |
| Approach: | They propose a multi-turn grounding-based policy optimization framework that enables LMMs to iteratively focus on key visual regions by automatically cropping sub-images based on model-predicted grounding coordinates within a multiple-turn conversation framework. |
| Outcome: | The proposed framework improves on Qwen2.5-VL-7B with 21K samples and surpasses OpenAI’s o1 and GPT-4o models on the out-of-distribution (OOD) V* Bench. |
Towards Better Entity Linking with Multi-View Enhanced Distillation (2023.acl-long)
Copied to clipboard
Yi Liu, Yuan Tian, Jianxun Lian, Xinlong Wang, Yanan Cao, Fang Fang, Wen Zhang, Haizhen Huang, Weiwei Deng, Qi Zhang
| Challenge: | Entity linking is a fundamental task in Natural Language Processing (NLP), connecting mentions within unstructured contexts to their corresponding entities in a Knowledge Base (KB). |
| Approach: | They propose a dual-encoder framework that can efficiently match mentions to two-encoding frameworks by a global-view. |
| Outcome: | The proposed framework achieves state-of-the-art on several entity linking benchmarks. |
Calibrating LLM-Based Evaluator (2024.lrec-main)
Copied to clipboard
Yuxuan Liu, Tianchi Yang, Shaohan Huang, Zihan Zhang, Haizhen Huang, Furu Wei, Weiwei Deng, Feng Sun, Qi Zhang
| Challenge: | Existing models for large language models lack the ability to calibrate their outputs towards human preference. |
| Approach: | They propose a multi-stage, gradient-free approach to calibrate an LLM-based evaluator toward human preference. |
| Outcome: | The proposed approach improves correlation with expert evaluation on multiple text quality evaluation datasets. |
UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation (2023.emnlp-main)
Copied to clipboard
Daixuan Cheng, Shaohan Huang, Junyu Bi, Yuefeng Zhan, Jianfeng Liu, Yujing Wang, Hao Sun, Furu Wei, Weiwei Deng, Qi Zhang
| Challenge: | Large Language Models (LLMs) have impressive capabilities but need for task-specific prompt engineering can hinder their generalization. |
| Approach: | They propose a lightweight and versatile retriever that automatically retrieves prompts for a given zero-shot task input. |
| Outcome: | The proposed model is universally applicable across tasks and models . it mitigates hallucination problem in chatGPT, and it improves even the strongest LLMs. |
MetaWeighting: Learning to Weight Tasks in Multi-Task Learning (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing task weighting methods assign weights only based on training losses, while ignoring the gap between the training loss and generalization loss. |
| Approach: | They propose a task weighting algorithm which automatically weights the tasks via a learning-to-learn paradigm and a multi-task text classification paradigm. |
| Outcome: | Extensive experiments show that the proposed method outperforms existing methods in multi-task text classification. |
Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated exceptional proficiency in language-related tasks, but their deployment poses significant memory and storage requirements. |
| Approach: | They propose a method that optimizes rounding values and weight clipping within 200 steps. |
| Outcome: | The proposed method achieves exceptional results across 2 to 4 bits while maintaining low tuning costs and avoiding additional inference overhead. |
MAIR: A Massive Benchmark for Evaluating Instructed Retrieval (2024.emnlp-main)
Copied to clipboard
Weiwei Sun, Zhengliang Shi, Wu Long, Lingyong Yan, Xinyu Ma, Yiding Liu, Min Cao, Dawei Yin, Zhaochun Ren
| Challenge: | Existing IR benchmarks focus on a limited scope of tasks, making them insufficient for evaluating the latest IR models. |
| Approach: | They propose a multi-task instruction-tuned IR benchmark that includes 126 distinct IR tasks across 6 domains. |
| Outcome: | The proposed model performs better on instruction-tuned models than non-instruction-tunned models on MAIR. |