Papers by Zhenghao Liu

RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework (2025.acl-long)

Challenge:	Existing approaches to multimodal representation learning focus on directional alignment and embedding magnitudes (L2-norm) however, these methods often fail to account for the intrinsic role of L2-norm in the contrastive process.
Approach:	They propose a plug-and-play framework that optimizes L2-norm alignment and Directional consistency jointly.
Outcome:	The proposed framework achieves consistent and significant performance gains over established baselines across 95 tasks using UniIR and VLM2Vec-V2 frameworks.

Copied to clipboard

Kunlun Zhu, Yifan Luo, Dingling Xu, Yukun Yan, Zhenghao Liu, Shi Yu, Ruobing Wang, Shuo Wang, Yishan Li, Nan Zhang, Xu Han, Zhiyuan Liu, Maosong Sun

Text Generation

Challenge:	Existing evaluation metrics for RAG systems are lacking due to high costs of data construction and lack of factual accuracy.
Approach:	They propose a framework to evaluate RAG systems in specialized scenarios . they propose three new metrics to evaluate LLM-generated responses .
Outcome:	The proposed framework outperforms zero-shot and one-shot methods in terms of clarity, safety, conformity, and richness of generated samples.

Fine-grained Fact Verification with Kernel Graph Attention Network (2020.acl-main)

Copied to clipboard

Zhenghao Liu, Chenyan Xiong, Maosong Sun, Zhiyuan Liu

Responsible & Trustworthy NLP

Challenge:	Existing methods for fact verification are based on dot-product attentions, but kernel-based attentions focus more on relevant evidence sentences and meaningful clues in the evidence graph.
Approach:	They propose a kernel-based attention network which conducts more fine-grained fact verification with kernel-basic attentions.
Outcome:	The proposed task achieves a 70.38% FEVER score and significantly outperforms existing fact verification models on FEVER, a large-scale benchmark for fact verification.

Coreferential Reasoning Learning for Language Representation (2020.emnlp-main)

Copied to clipboard

Deming Ye, Yankai Lin, Jiaju Du, Zhenghao Liu, Peng Li, Maosong Sun, Zhiyuan Liu

MCTS: A Multi-Reference Chinese Text Simplification Dataset (2024.lrec-main)

Challenge:	Existing language representation models cannot explicitly handle coreference, which is essential to the coherent understanding of the whole discourse.
Approach:	They propose a language representation model that captures coreferential relations in context.
Outcome:	The proposed model can achieve significant improvements on downstream NLP tasks while maintaining comparable performance to baseline models on other common NLP task.

Copied to clipboard

Ruining Chong, Luming Lu, Liner Yang, Jinran Nie, Zhenghao Liu, Shuo Wang, Shuhan Zhou, Yaoxin Li, Erhong Yang

Text Generation

Challenge:	Existing studies on text simplification systems have focused on unsupervised methods due to the limited evaluation data in language and domain.
Approach:	They propose a Chinese text simplification dataset that provides a detailed analysis and an annotation process.
Outcome:	The proposed dataset evaluates the performance of unsupervised methods and advanced large language models.

Revealing the Attention Floating Mechanism in Masked Diffusion Models (2026.findings-acl)

Copied to clipboard

Xin Dai, Pengcheng Huang, Zhenghao Liu, Shuo Wang, Yukun Yan, Chaojun Xiao, Yu Gu, Ge Yu, Maosong Sun

PersLLM: A Personified Training Approach for Large Language Models (2025.findings-emnlp)

Challenge:	Masked diffusion models (MDMs) leverage bidirectional attention and a denoising process.
Approach:	They investigate the attention behaviors of Masked diffusion models by revealing the phenomenon of Attention Floating.
Outcome:	The proposed model doubles the performance of autoregressive models in knowledge-intensive tasks.

Copied to clipboard

Zheni Zeng, Jiayi Chen, Huimin Chen, Yukun Yan, Yuxuan Chen, Zhenghao Liu, Zhiyuan Liu, Maosong Sun

Leveraging Prefix Transfer for Multi-Intent Text Revision (2023.acl-short)

Challenge:	Large language models exhibit human-like intelligence, enabling them to simulate human behavior and support various applications that require both humanized communication and extensive knowledge reserves.
Approach:	They propose a framework for better data construction and model tuning to unlock the potential of LLM personification by using Chain-of-Thought prompting and anti-induction.
Outcome:	The proposed framework improves data construction and model tuning for insufficient data usage and rigid behavior patterns.

Copied to clipboard

Ruining Chong, Cunliang Kong, Liu Wu, Zhenghao Liu, Ziye Jin, Liner Yang, Yange Fan, Hanghang Fan, Erhong Yang

Challenge:	Text revision is a necessary process to improve text quality.
Approach:	They propose a multi-intent text revision system that can revise texts without explicit intent annotation.
Outcome:	The proposed system outperforms baselines on the IteraTeR dataset and significantly improves the SARI score with more than 3% improvement.

Adapting Open Domain Fact Extraction and Verification to COVID-FACT through In-Domain Language Modeling (2020.findings-emnlp)

Copied to clipboard

Zhenghao Liu, Chenyan Xiong, Zhuyun Dai, Si Sun, Maosong Sun, Zhiyuan Liu

KBAlign: Efficient Self Adaptation on Specific Textual Knowledge Bases (2025.findings-emnlp)

Challenge:	Existing methods to verify scientifically false online information are limited by the lack of training data in the scientific domain.
Approach:	They propose an in-domain language modeling method for fact extraction and verification systems . they use SCIFACT to extract scientifically false online information .
Outcome:	The proposed method improves accuracy 30% on SCIFACT dataset . state-of-the-art model achieves only 46.6% precision, which is hard to be trusted for users.

Copied to clipboard

Zheni Zeng, Yuxuan Chen, Shi Yu, Ruobing Wang, Yukun Yan, Zhenghao Liu, Shuo Wang, Xu Han, Zhiyuan Liu, Maosong Sun

Judge as A Judge: Improving the Evaluation of Retrieval-Augmented Generation through the Judge-Consistency of Large Language Models (2025.findings-acl)

Challenge:	Existing methods for retrieval-augmented generation (RAG) are limited and fine-tuning incurs prohibitive costs of external signals.
Approach:	They propose a self-supervised framework that enhances RAG systems through efficient model adaptation.
Outcome:	The proposed framework achieves 90% of the performance gain obtained through GPT-4-supervised adaptation while relying entirely on self-annotation of much smaller models.

Copied to clipboard

Shuliang Liu, Xinze Li, Zhenghao Liu, Yukun Yan, Cheng Yang, Zheni Zeng, Zhiyuan Liu, Maosong Sun, Ge Yu

InsightEval: An Expert-Curated Benchmark for Assessing Insight Discovery in LLM-Driven Data Agents (2026.findings-acl)

Challenge:	Existing evaluation metrics cannot fairly evaluate the outputs of RAG models during training and evaluation.
Approach:	They propose a method which prompts LLMs to generate different judgments based on various combinations of judgment dimensions and utilizes the judge-consistency to evaluate these judgments.
Outcome:	The proposed method generates more accurate evaluations for RAG models across different RAG model and datasets.

Copied to clipboard

Zhenghao Zhu, Yuanfeng Song, Xing Chen, Chengzhong Liu, Cui Yakun, Caleb Chen Cao, Sirui Han, Yike Guo

Challenge:	Existing frameworks for data analysis and insight exploration are lacking in terms of benchmarks . existing frameworks suffer from format inconsistencies, poorly conceived objectives, and redundant insights.
Approach:	They propose a data-curation pipeline to construct a new dataset named InsightEval.
Outcome:	The proposed benchmarks highlight prevailing challenges in automated insight discovery and raise key findings to guide future research.

Enhancing Long-Chain Reasoning Distillation through Error-Aware Self-Reflection (2026.findings-acl)

Copied to clipboard

Zhuoyang Wu, Xinze Li, Zhenghao Liu, Yukun Yan, Zhiyuan Liu, Minghe Yu, Cheng Yang, Yu Gu, Ge Yu, Maosong Sun

DeepNote: Note-Centric Deep Retrieval-Augmented Generation (2025.findings-emnlp)

Challenge:	Existing studies treat SLMs as student models and use long-form Chains-of-Thought (CoTs) as supervision signals for Supervised Fine-Tuning (SFT). Existing research focuses on distilling reasoning ability from LLMs to enhance the mathematical reasoning performance of small-scale models.
Approach:	They propose a framework that refines teacher CoTs through an error-aware reflection process to enable the student model to construct more tailored teacher Cots.
Outcome:	Experiments on multiple mathematical reasoning benchmarks show that ORION improves performance by more than 2% over all baselines.

Copied to clipboard

Ruobing Wang, Qingfei Zhao, Yukun Yan, Daren Zha, Yuxuan Chen, Shi Yu, Zhenghao Liu, Yixuan Wang, Shuo Wang, Xu Han, Zhiyuan Liu, Maosong Sun

MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization (2024.findings-acl)

Challenge:	. - (EN)
Approach:	. - (EN)
Outcome:	. - (EN)

Copied to clipboard

Zhiyu Yang, Zihan Zhou, Shuo Wang, Xin Cong, Xu Han, Yukun Yan, Zhenghao Liu, Zhixing Tan, Pengyuan Liu, Dong Yu, Zhiyuan Liu, Xiaodong Shi, Maosong Sun

Structure-Aware Language Model Pretraining Improves Dense Retrieval on Structured Data (2023.findings-acl)

Challenge:	Scientific data visualization is an essential process in research, but its use of large language models remains unexplored.
Approach:	They propose a model-agnostic LLM agent framework to automate scientific data visualization tasks.
Outcome:	The proposed framework improves performance of commercial and open-source models.

Copied to clipboard

Xinze Li, Zhenghao Liu, Chenyan Xiong, Shi Yu, Yu Gu, Zhiyuan Liu, Ge Yu

INTERVENOR: Prompting the Coding Ability of Large Language Models with the Interactive Chain of Repair (2024.findings-acl)

Challenge:	Structure Aware Dense Retrieval (SANTA) model encodes user queries and structured data in one universal embedding space for retrieving structured data.
Approach:	They propose to use structured data and unstructured data to encode queries and structured data in one universal embedding space for retrieving structured data.
Outcome:	The proposed model achieves state-of-the-art on code search and product search and conducts convincing results in the zero-shot setting.

Copied to clipboard

Hanbin Wang, Zhenghao Liu, Shuo Wang, Ganqu Cui, Ning Ding, Zhiyuan Liu, Ge Yu

MetaMem: Evolving Meta-Memory for Knowledge Utilization through Self-Reflective Symbolic Optimization (2026.findings-acl)

Challenge:	Experimental results show that INTERVENOR surpasses baseline models, exhibiting improvements of approximately 18% and 4.3% over GPT-3.5 in code generation and code translation tasks.
Approach:	They propose a system that prompts Large Language Models to play distinct roles during the code repair process, functioning as both a Code Learner and a code teacher.
Outcome:	The proposed system surpasses baseline models in code generation and code translation tasks and improves on syntax errors and assertion errors.

Copied to clipboard

Haidong Xin, Xinze Li, Zhenghao Liu, Yukun Yan, Shuo Wang, Cheng Yang, Yu Gu, Ge Yu, Maosong Sun

ClueAnchor: Clue-Anchored Knowledge Reasoning Exploration and Optimization for Retrieval-Augmented Generation (2025.findings-emnlp)

Challenge:	Existing memory systems can support long-horizon human-LLM interactions by persisting historical interactions beyond limited context windows.
Approach:	They propose a framework that augments memory systems with a self-evolving meta-memory . meta-meso is iteratively distilling transferable knowledge utilization experiences . results show MetaMem outperforms strong baselines by over 3.6% .
Outcome:	The proposed framework outperforms baselines by over 3.6% in the long-horizon human-LLM interaction.

Copied to clipboard

Hao Chen, Yukun Yan, Sen Mei, Wanxiang Che, Zhenghao Liu, Qi Shi, Xinze Li, Yuchun Fan, Pengcheng Huang, Qiushi Xiong, Zhiyuan Liu, Maosong Sun

Mitigating Judgment Preference Bias in Large Language Models through Group-Based Polling (2026.findings-acl)

Challenge:	Existing RAG systems often underutilize the retrieved documents, authors say . they fail to extract and integrate key clues needed to support faithful and interpretable reasoning .
Approach:	a new framework extracts key clues from retrieved content and generates multiple reasoning paths . the framework optimizes the model by selecting the most appropriate reasoning path .
Outcome:	Experiments show that ClueAnchor outperforms baseline RAG frameworks in completeness and robustness.

Copied to clipboard

Shuliang Liu, Zhipeng Xu, Zhenghao Liu, Yukun Yan, Minghe Yu, Yu Gu, Chong Chen, Huiyuan Xie, Ge Yu

ThinkNote: Enhancing Knowledge Integration and Utilization of Large Language Models via Constructivist Cognition Modeling (2026.findings-eacl)

Challenge:	Large Language Models (LLMs) are used as automatic evaluators to provide accurate and reliable assessments.
Approach:	They propose a framework that integrates LLM-based judgment models into a multi-agent system and simulates the interactive client-server polling mechanism.
Outcome:	The proposed framework outperforms supervised models trained on annotated judgment data while requiring no human-labeled annotations.

Copied to clipboard

Zhipeng Xu, Zhenghao Liu, Yukun Yan, Shuo Wang, Shi Yu, Zheni Zeng, Chaojun Xiao, Zhiyuan Liu, Ge Yu, Chenyan Xiong

Few-Shot Text Ranking with Meta Adapted Synthetic Weak Supervision (2021.acl-long)

Challenge:	Large Language Models (LLMs) exhibit suboptimal behaviors and inconsistencies when exposed to unfamiliar external information, underscoring their limitations in effectively leveraging such knowledge.
Approach:	They propose a framework that enhances the external knowledge utilization of Large Language Models through a two-stage constructivist cognitive modeling process.
Outcome:	The proposed framework achieves a 10% improvement over baseline methods on various question-answering benchmarks.

Copied to clipboard

Si Sun, Yingzhuo Qian, Zhenghao Liu, Chenyan Xiong, Kaitao Zhang, Jie Bao, Zhiyuan Liu, Paul Bennett

Low-Resource NLP

Challenge:	Neural information retrieval models have shown advanced results in many ranking scenarios where massive relevance labels or clickthrough data are available.
Approach:	They propose a domain adaptive learning method that generalizes Neu-IR models from label-rich source domains to few-shot target domains.
Outcome:	The proposed method improves the few-shot ranking accuracy of Neu-IR models on three TREC benchmarks in the web, news, and biomedical domains.

Fusion-in-T5: Unifying Variant Signals for Simple and Effective Document Ranking with Attention Fusion (2024.lrec-main)

Copied to clipboard

Shi Yu, Chenghao Fan, Chenyan Xiong, David Jin, Zhiyuan Liu, Zhenghao Liu

UNIKIE-BENCH: Benchmarking Large Multimodal Models for Key Information Extraction in Visual Documents (2026.acl-long)

Challenge:	Current document ranking pipelines involve multiple ranking layers to integrate different information step-by-step.
Approach:	They propose a novel re-ranker Fusion-in-T5 which integrates text matching information, ranking features, and global document information into one single unified model via templated-based input and global attention.
Outcome:	The proposed model significantly improves ranking performance over complex cascade pipelines.

Copied to clipboard

Yifan Ji, Zhipeng Xu, Zhenghao Liu, Zulong Chen, Qian Zhang, Zhibo Yang, Junyang Lin, Yu Gu, Ge Yu, Maosong Sun

Information Extraction & Text Mining

Challenge:	Recent Large Multimodal Models (LMMs) have shown promising potential for performing end-to-end KIE directly from document images.
Approach:	They propose a benchmark to evaluate the performance of Large Multimodal Models (LMMs) using a constrained-category KIE track and an open-categorical KIE Track.
Outcome:	Experiments on 15 state-of-the-art LMMs show performance degradation under diverse schema definitions, long-tail key fields, and complex layouts, along with pronounced performance disparities across different document types and scenarios.

Entity-Duet Neural Ranking: Understanding the Role of Knowledge Graph Semantics in Neural Information Retrieval (P18-1)

Copied to clipboard

Zhenghao Liu, Chenyan Xiong, Maosong Sun, Zhiyuan Liu

Chunks as Arms: Multi-Armed Bandit-Guided Sampling for Long-Context LLM Preference Optimization (2026.acl-long)

Challenge:	Entity-oriented search and neural-IR push the boundary of search engines from two different aspects.
Approach:	They propose an Entity-Duet Neural Ranking Model which integrates knowledge graphs into neural search systems.
Outcome:	The proposed model improves generalization ability of neural ranking models on a commercial search log.

Copied to clipboard

Shaohua Duan, Pengcheng Huang, Xinze Li, Zhenghao Liu, Xiaoyuan Yi, Yukun Yan, Shuo Wang, Yu Gu, Ge Yu, Maosong Sun

ReCUT: Balancing Reasoning Length and Accuracy in LLMs via Stepwise Trails and Preference Optimization (2025.findings-emnlp)

Challenge:	Recent studies have explored fine-tuning Large Language Models with synthetic data to enhance their long-context capabilities.
Approach:	They propose a framework that leverages a Multi-Armed Bandit rollout strategy to identify the most informative chunks from the given long context for sampling high-quality and diverse responses.
Outcome:	The proposed framework achieves 4% improvement on long-context reasoning benchmarks on Llama and Qwen.

Copied to clipboard

Zhensheng Jin, Xinze Li, Yifan Ji, Chunyi Peng, Zhenghao Liu, Qi Shi, Yukun Yan, Shuo Wang, Furong Peng, Ge Yu

HqeKV: Towards Hybrid Quantization and Eviction for KV Cache in Long-Context LLM Inference (2026.findings-acl)

Challenge:	Existing methods to train LLMs suffer from overthinking, leading to lengthy reasoning traces . Existing approaches to train large language models suffer from this problem .
Approach:	They propose a method to combine multiple reasoning chains for training LLMs . they use stepwise exploration and long-short switched sampling to evaluate reasoning paths .
Outcome:	The proposed method reduces reasoning lengths by approximately 30-50% . it also maintains or improves reasoning accuracy compared to baselines .

Copied to clipboard

He Wang, Yu Gu, Fangfang Li, Zhigang Wang, Zhenghao Liu, Ning Wang, Xiaohua Li, Ge Yu

Challenge:	autoregressive inference requires repeated computation across transformer layers.
Approach:	They propose a hybrid compression framework built on both quantization and eviction . they propose varying importance metric and flexible conversion policies to reduce memory overhead .
Outcome:	The proposed framework outperforms state-of-the-art methods under memory constraints.

RankCoT: Refining Knowledge for Retrieval-Augmented Generation through Ranking Chain-of-Thoughts (2025.acl-long)

Copied to clipboard

Mingyan Wu, Zhenghao Liu, Yukun Yan, Xinze Li, Shi Yu, Zheni Zeng, Yu Gu, Ge Yu

Text Generation

Challenge:	Retrieval-Augmented Generation (RAG) models enable Large Language Models to access external knowledge.
Approach:	They propose a knowledge refinement method that incorporates reranking signals to generate CoT-based summarization based on query and retrieval documents.
Outcome:	RankCoT generates CoT-based summarization based on query and all retrieval documents . Rank CoT incorporates a self-reflection mechanism that refines the outputs .

LayerNorm Induces Recency Bias in Transformer Decoders (2026.findings-acl)

Copied to clipboard

Junu Kim, Xiao Liu, Zhenghao Lin, Lei Ji, Yeyun Gong, Edward Choi

Dimension Reduction for Efficient Dense Retrieval via Conditional Autoencoder (2022.emnlp-main)

Challenge:	Existing studies show that stacking causal self-attention layers alone induces a positional bias in attention scores toward earlier tokens, but this differs from the bias toward later tokens observed in Transformer decoders, known as recency bias.
Approach:	They propose to stack causal self-attention layers and layer norm to induce recency bias in Transformer decoders by analyzing the interaction between causal self and other architectural components.
Outcome:	The proposed method provides new theoretical insights into how positional information interacts with architectural components and suggests improvements in positional encoding strategies.

Copied to clipboard

Zhenghao Liu, Han Zhang, Chenyan Xiong, Zhiyuan Liu, Yu Gu, Xiaohua Li

CheckRLM: Effective Knowledge–Thought Coherence Checking in Retrieval-Augmented Reasoning (2026.acl-long)

Challenge:	Existing work reserves the principle dimensions of query and document embeddings for building more efficient retrieval systems.
Approach:	They propose to use Conditional Autoencoder to compress high-dimensional embeddings to maintain the same embeddable distribution and better recover ranking features.
Outcome:	The proposed algorithm achieves comparable ranking performance with its teacher model and makes the retrieval system more efficient.

Copied to clipboard

Dingling Xu, Ruobing Wang, Qingfei Zhao, Yukun Yan, Zhichun Wang, Daren Zha, Shi Yu, Zhenghao Liu, Shuo Wang, Xu Han, Maosong Sun

Neural Quality Estimation with Multiple Hypotheses for Grammatical Error Correction (2021.naacl-main)

Challenge:	Reasoning Language Models (RLMs) have improved performance on complex tasks by extending the reasoning chain, but they are prone to factual errors, especially in knowledge-intensive tasks.
Approach:	They propose a framework that improves the reliability of the reasoning process by timely checking and correcting factual errors.
Outcome:	The proposed framework outperforms baselines and shows that it mitigates error accumulation with lower costs.

Copied to clipboard

Zhenghao Liu, Xiaoyuan Yi, Maosong Sun, Liner Yang, Tat-Seng Chua

Syntactic Text Processing

Challenge:	Existing GEC models produce spurious corrections or fail to detect lots of errors.
Approach:	They propose a neural network for GEC quality estimation with multiple hypotheses . VERNet establishes interactions among hypothese based on reasoning graph .
Outcome:	The proposed model achieves state-of-the-art grammatical error detection performance and best quality estimation results on four GEC datasets.

Enhancing Legal Case Retrieval via Scaling High-quality Synthetic Query-Candidate Pairs (2024.emnlp-main)

Copied to clipboard

Cheng Gao, Chaojun Xiao, Zhenghao Liu, Huimin Chen, Zhiyuan Liu, Maosong Sun

ExpandR: Teaching Dense Retrievers Beyond Queries with LLM Guidance (2025.emnlp-main)

Challenge:	Existing studies focus on case-to-case retrieval using lengthy queries, which does not match real-world scenarios.
Approach:	They propose a method to construct query-candidate pairs and build the largest LCR dataset to date, LEAD.
Outcome:	Experimental results show that the method can provide ample training signals for LCR models.

Copied to clipboard

Sijia Yao, Pengcheng Huang, Zhenghao Liu, Yu Gu, Yukun Yan, Shi Yu, Ge Yu

TIAGE: A Benchmark for Topic-Shift Aware Dialog Modeling (2021.findings-emnlp)

Challenge:	Existing methods for enhancing dense retrieval with query augmentation ignore the alignment between generation and ranking objectives.
Approach:	They propose a unified LLM-augmented dense retrieval framework that jointly optimizes both the LLM and the retriever.
Outcome:	Experimental results show that ExpandR outperforms strong baselines, achieving more than 5% improvement in retrieval performance.

Copied to clipboard

Huiyuan Xie, Zhenghao Liu, Chenyan Xiong, Zhiyuan Liu, Ann Copestake

Dialogue Systems & Conversational Agents

Challenge:	Existing dialog models can generate on-topic utterances but struggle to proactively switch topics.
Approach:	They propose a topic-shift aware dialog benchmark based on human topic shift annotations.
Outcome:	The proposed benchmark enables chatbots to generate topic-shift responses while still struggling to decide when to change topic.

Empirical Analysis of Decoding Biases in Masked Diffusion Models (2026.acl-long)

Copied to clipboard

Pengcheng Huang, Tianming Liu, Zhenghao Liu, Yukun Yan, Shuo Wang, Tong Xiao, Zulong Chen, Maosong Sun

Long-Chain Reasoning Distillation via Adaptive Prefix Alignment (2026.acl-long)

Challenge:	Existing MDMs employ uncertainty-based decoding strategies that limit their reasoning ability and ultimately degrade generation quality.
Approach:	They propose a framework that regularizes uncertainty-based decoding by incorporating two complementary priors to shape global decoding trajectories and promote content informativeness.
Outcome:	The proposed framework outperforms existing decoding strategies by more than 7% while achieving comparable performance to autoregressive models of similar parameter scales.

Copied to clipboard

Zhenghao Liu, Zhuoyang Wu, Xinze Li, Yukun Yan, Shuo Wang, Zulong Chen, Yu Gu, Ge Yu, Maosong Sun

Lang2Act: Fine-Grained Visual Reasoning through Self-Emergent Linguistic Toolchains (2026.findings-acl)

Challenge:	Large Language Models (LLMs) have demonstrated remarkable reasoning capabilities, especially in solving complex mathematical problems.
Approach:	They propose a framework that exploits teacher CoTs for distillation through adaptive prefix alignment.
Outcome:	The proposed framework outperforms baseline models on multiple mathematical reasoning benchmarks by over 3%.

Copied to clipboard

Yuqi Xiong, Chunyi Peng, Zhipeng Xu, Zhenghao Liu, Zulong Chen, Yukun Yan, Shuo Wang, Yu Gu, Ge Yu

UltraLink: An Open-Source Knowledge-Enhanced Multilingual Supervised Fine-tuning Dataset (2024.acl-long)

Challenge:	Existing frameworks depend on rigid, pre-defined external tools to extend perceptual capabilities of VLMs.
Approach:	They propose a framework that leverages self-emergent linguistic toolchains to enhance visual perception and reasoning.
Outcome:	The proposed framework improves the visual perception capabilities of large language models by incorporating external visual documents to address a given query.

Copied to clipboard

Haoyu Wang, Shuo Wang, Yukun Yan, Xujia Wang, Zhiyu Yang, Yuzhuang Xu, Zhenghao Liu, Liner Yang, Ning Ding, Xu Han, Zhiyuan Liu, Maosong Sun

Towards Efficient and Effective Diffusion Language Model Inference via Semantic-Aware Adaptive Denoising (2026.acl-long)

Challenge:	Open-source large language models (LLMs) have gained strength across diverse fields, but the majority of studies focus on English.
Approach:	They propose a knowledge-grounded data augmentation approach to elicit more language-specific knowledge of LLMs by enhancing their ability to serve users from different countries.
Outcome:	The proposed method can prune the language-agnostic supervised fine-tuning dataset without any performance degradation.

Copied to clipboard

Fan Li, Yu Gu, Zhigang Wang, Fangling Leng, Zhenghao Liu, Ge Yu

Grounded Conversation Generation as Guided Traverses in Commonsense Knowledge Graphs (2020.acl-main)

Challenge:	Existing acceleration works cannot accurately detect semantically stabilized tokens and then skip computation, leading to sub-optimal speedup in practice.
Approach:	They propose a semantic-aware adaptive denoising framework that encodes scalar confidence scores into an evolution-awful feature vector and clusters vectors proactively and adaptively identify semantically converged tokens.
Outcome:	The proposed framework outperforms the SOTA competitor in speed and quality . it can detect semantically stabilized tokens and skip computation, resulting in sub-optimal speedup .

Copied to clipboard

Houyu Zhang, Zhenghao Liu, Chenyan Xiong, Zhiyuan Liu

Dialogue Systems & Conversational Agents

Challenge:	Existing models that generate natural language responses for conversations degenerate dull and repetitive contents, leading to off-topic and useless responses.
Approach:	They propose a conversation generation model which leverages commonsense knowledge graphs to explicitly model conversation flows by grounding conversations to the concept space.
Outcome:	Experiments on Reddit conversations show that the proposed model generates more semantic and informative responses while using 70% fewer parameters.

COAST: Enhancing the Code Debugging Ability of LLMs through Communicative Agent Based Data Synthesis (2025.findings-naacl)

Copied to clipboard

Weiqing Yang, Hanbin Wang, Zhenghao Liu, Xinze Li, Yukun Yan, Shuo Wang, Yu Gu, Minghe Yu, Zhiyuan Liu, Ge Yu

Toolink: Linking Toolkit Creation and Using through Chain-of-Solving on Open-Source Model (2024.naacl-long)

Challenge:	Existing code debugging benchmarks focus on the Code Repair stage of the code generation process.
Approach:	They propose a framework to evaluate the debugging abilities of large language models by emulating the human debug process.
Outcome:	The proposed framework outperforms human-curated and GPT-4-generated training data, enabling 7B-scale LLMs to achieve comparable debugging performance to GPT-3.5.

Copied to clipboard

Cheng Qian, Chenyan Xiong, Zhenghao Liu, Zhiyuan Liu

Competition-Level Problems are Effective LLM Evaluators (2024.findings-acl)

Challenge:	Large Language Models (LLMs) have made significant progress in utilizing tools, but their closed-source nature and high inference costs pose limitations on their adaptability.
Approach:	They propose a framework that performs task-solving by first creating a toolkit and then integrating the planning and calling of tools through a chain-of-solve approach.
Outcome:	The proposed model performs task-solving by harnessing Toolink's creativity and CoS ability on ChatGPT and finetunes the LLaMA-7B model.

Copied to clipboard

Yiming Huang, Zhenghao Lin, Xiao Liu, Yeyun Gong, Shuai Lu, Fangyu Lei, Yaobo Liang, Yelong Shen, Chen Lin, Nan Duan, Weizhu Chen