Papers by Pengfei Li

Beyond Surface-Level Pattern Trap: LLM Agents for Faster and Smarter Cross-Architecture Code Migration (2026.findings-acl)

Challenge:	Existing knowledge conflicts in RALMs can ensnare them in a tug-of-war between knowledge and evidence, limiting their practical applicability.
Approach:	They propose a method called Conflict-Disentangle Contrastive Decoding (CD2) to better calibrate the model’s confidence.
Outcome:	The proposed method can resolve knowledge conflicts in large language models with the help of conflict-disentangle contrast decoding (CD2) .

Copied to clipboard

Weijia Li, KE Gao, Pengfei Chen, Jiajie Li, Xinyu Wang, Yiran Le, Yize Wu, Ling Li

Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Survey (2025.emnlp-main)

Challenge:	cross-architecture code migration is a resource-intensive and errorprone task.
Approach:	a framework for cross-architecture code migration is proposed to decouple implementation details through functional mining and code refactoring.
Outcome:	a new framework improves performance and correctness over state-of-the-art frameworks on OpenCV migration tasks.

Copied to clipboard

Tianxin Xie, Yan Rong, Pengfei Zhang, Wenwu Wang, Li Liu

Multimodality

Challenge:	Text-to-speech (TTS) has advanced from generating natural-sounding speech to enabling fine-grained control over speech attributes.
Approach:	They provide a review of controllable TTS methods from traditional control techniques to emerging approaches using natural language prompts.
Outcome:	The proposed methods are based on models, strategies, and features, and summarize challenges, datasets, and evaluations.

Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Models (2024.findings-acl)

Copied to clipboard

Zhuoran Jin, Pengfei Cao, Hongbang Yuan, Yubo Chen, Jiexin Xu, Huaijun Li, Xiaojian Jiang, Kang Liu, Jun Zhao

Focus on Your Question! Interpreting and Mitigating Toxic CoT Problems in Commonsense Reasoning (2024.acl-long)

Challenge:	Existing methods to expand internal memory boundaries of language models by providing external context can often conflict, leading to knowledge conflicts.
Approach:	They propose a method that prunes conflicting attention heads without updating model parameters.
Outcome:	The proposed method can flexibly control eight LMs to use internal memory or external context without updating model parameters.

Copied to clipboard

Jiachun Li, Pengfei Cao, Chenhao Wang, Zhuoran Jin, Yubo Chen, Daojian Zeng, Kang Liu, Jun Zhao

Why Does New Knowledge Create Messy Ripple Effects in LLMs? (2024.emnlp-main)

Challenge:	Large language models exhibit high-level commonsense reasoning abilities, especially with enhancement methods like Chain-of-Thought (CoT).
Approach:	They propose a chain-of-thought-like method to elicit models' potential abilities to generate rationales and answers that are based on attribution tracing and causal tracers to probe the internal working mechanism of the LLM.
Outcome:	The proposed method eliminates Toxic CoT problems and improves the model’s overall commonsense reasoning performance by 5.5%.

Copied to clipboard

Jiaxin Qin, Zixuan Zhang, Chi Han, Pengfei Yu, Manling Li, Heng Ji

LLM-Based Human-Agent Collaboration and Interaction Systems: A Survey (2026.findings-acl)

Challenge:	Existing research has focused on post-training knowledge editing (KE) for language models to ensure that knowledge remains accurate and up-to-date.
Approach:	They propose to use a GradSim indicator to detect when and why updated knowledge ripples in language models.
Outcome:	The proposed indicator GradSim shows that LMs that fail to handle ripple effects have low GradSIM.

Copied to clipboard

Henry Peng Zou, Wei-Chieh Huang, Yaozu Wu, Jizhou Guo, Yankai Chen, Chunyu Miao, Hoang H Nguyen, Yue Zhou, Weizhi Zhang, Liancheng Fang, Hanrong Zhang, Fangxin Wang, Pengfei Zhang, Langzhou He, Yangning Li, Dongyuan Li, Renhe Jiang, Philip S. Yu

Dissecting Human and LLM Preferences (2024.acl-long)

Challenge:	Recent advances in large language models (LLMs) have sparked growing interest in building fully autonomous agents.
Approach:	They propose to integrate human-provided information, feedback, or control into the agent system to enhance system performance, reliability, and safety.
Outcome:	The proposed systems improve system performance, reliability, and safety by integrating human-provided information, feedback, or control into the agent system.

Copied to clipboard

Junlong Li, Fan Zhou, Shichao Sun, Yikai Zhang, Hai Zhao, Pengfei Liu

EVEDIT: Event-based Knowledge Editing for Deterministic Knowledge Propagation (2024.emnlp-main)

Challenge:	a recent study shows that human and Large Language Model preferences are important for model fine-tuning and evaluation.
Approach:	They dissect the preferences of human and 32 different Large Language Models to understand their quantitative composition.
Outcome:	The proposed model is compared with 32 different large language models using real-world user-model conversations.

Copied to clipboard

Jiateng Liu, Pengfei Yu, Yuji Zhang, Sha Li, Zixuan Zhang, Ruhi Sarikaya, Kevin Small, Heng Ji

LongCLI-Bench: A Preliminary Benchmark and Study for Long-horizon Agentic Programming in Command-Line Interfaces (2026.findings-acl)

Challenge:	Existing knowledge editing approaches only operate on (subject, relation, object) triple . current methods are limited to (substance, relation) triple, causing low confidence in their answers.
Approach:	They propose a task of event-based knowledge editing that pairs facts with event descriptions to improve model confidence.
Outcome:	The proposed method improves model confidence by 55.6% while maintaining the naturalness of generation.

Copied to clipboard

Yukang Feng, Jianwen Sun, Zelai Yang, Jiaxin Ai, Chuanhao Li, Zizhen Li, Fanrui Zhang, Kang He, Rui Ma, Jifan Lin, Jie Sun, Yang Xiao, Sizhuo Zhou, Wenxiao Wu, Yiming Liu, Pengfei Liu, Shenglin Zhang, Kaipeng Zhang

Multimodality

Challenge:	Existing benchmarks for agentic programming in long-horizon command-line interface tasks are limited by short task horizons, data contamination from GitHub scraping, and a lack of fine-grained evaluation metrics.
Approach:	They propose a benchmark to evaluate agentic capabilities across long-horizon command-line interface tasks.
Outcome:	The proposed benchmarks cover four engineering categories: from scratch, feature addition, bug fixing, and refactoring.

Advancing Reasoning with Off-the-Shelf LLMs: A Semantic Structure Perspective (2025.findings-emnlp)

Copied to clipboard

Pengfei He, Zitao Li, Yue Xing, Yaliang Li, Jiliang Tang, Bolin Ding

Low-Resource NLP

Challenge:	Existing reasoning models suffer from hallucinations and unfaithfulness, whereas general LLMs perform suboptimal on complex tasks.
Approach:	They propose a structure analysis method that helps LLMs better understand the question structure and guide the problem-solving process.
Outcome:	The proposed method improves zero-shot performance on knowledge-intensive and mathematical tasks while demonstrating strong robustness against corrupted reasoning paths.

Certified Robustness to Word Substitution Attack with Differential Privacy (2021.naacl-main)

Copied to clipboard

Wenjie Wang, Pengfei Tang, Jian Lou, Li Xiong

Responsible & Trustworthy NLP

Challenge:	Recent studies have shown that adversarial examples can be easily fooled by DNNs, making the robustness and security of NLP models significantly important.
Approach:	They propose a differential privacy-based algorithm to achieve certified robustness against word substitution at- tacks in text classification via differential privacy.
Outcome:	The proposed model achieves higher accuracy and more than 30X efficiency improvement over existing defense algorithms.

LINKED: Eliciting, Filtering and Integrating Knowledge in Large Language Model for Commonsense Reasoning (2024.findings-emnlp)

Copied to clipboard

Jiachun Li, Pengfei Cao, Chenhao Wang, Zhuoran Jin, Yubo Chen, Kang Liu, Xiaojian Jiang, Jiexin Xu, Jun Zhao

Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models (2025.findings-acl)

Challenge:	Large language models (LLMs) often exhibit poor performance on knowledge-intensive tasks, such as commonsense reasoning.
Approach:	They propose a method to elicit, filter and integrate knowledge in large language models (LINKED) they propose 'reward model' to filter out noisy knowledge and 'take marginal consistent reasoning module'
Outcome:	The proposed method outperforms SOTA baselines on two commonsense reasoning tasks.

Copied to clipboard

Yingqian Cui, Pengfei He, Jingying Zeng, Hui Liu, Xianfeng Tang, Zhenwei Dai, Yan Han, Chen Luo, Jing Huang, Zhen Li, Suhang Wang, Yue Xing, Jiliang Tang, Qi He

RESIN-11: Schema-guided Event Prediction for 11 Newsworthy Scenarios (2022.naacl-demo)

Challenge:	Chain-of-Thought (CoT) reasoning has improved the performance of large language models (LLMs) however, the detailed reasoning process in CoT often incurs long generation times and high computational costs due to the inclusion of unnecessary steps.
Approach:	They propose a method to identify critical reasoning steps using perplexity as a measure of their importance.
Outcome:	The proposed method achieves a better balance between reasoning accuracy and efficiency of CoT.

Copied to clipboard

Xinya Du, Zixuan Zhang, Sha Li, Pengfei Yu, Hongwei Wang, Tuan Lai, Xudong Lin, Ziqi Wang, Iris Liu, Ben Zhou, Haoyang Wen, Manling Li, Darryl Hannan, Jie Lei, Hyounghun Kim, Rotem Dror, Haoyu Wang, Michael Regan, Qi Zeng, Qing Lyu, Charles Yu, Carl Edwards, Xiaomeng Jin, Yizhu Jiao, Ghazaleh Kazeminejad, Zhenhailong Wang, Chris Callison-Burch, Mohit Bansal, Carl Vondrick, Jiawei Han, Dan Roth, Shih-Fu Chang, Martha Palmer, Heng Ji

Exploring Memorization in Fine-tuned Language Models (2024.acl-long)

Challenge:	Existing methods for event prediction are incomplete and noisy.
Approach:	They propose to use news-related event schemas to extract newsworthy events . they build a demo website and include a video demonstrating the framework .
Outcome:	The proposed framework can be applied to a wide variety of newsworthy scenarios.

Copied to clipboard

Shenglai Zeng, Yaxin Li, Jie Ren, Yiding Liu, Han Xu, Pengfei He, Yue Xing, Shuaiqiang Wang, Jiliang Tang, Dawei Yin

Reformatted Alignment (2024.findings-emnlp)

Challenge:	Existing studies have shown that pre-trained langauge models tend to memorize and regenerate segments of their pre-training corpus when prompted appropriately.
Approach:	They conduct the first comprehensive analysis to explore language models’ memorization during fine-tuning across tasks.
Outcome:	The proposed analysis shows that memorization presents a strong disparity among different fine-tuning tasks.

Copied to clipboard

Run-Ze Fan, Xuefeng Li, Haoyang Zou, Junlong Li, Shwai He, Ethan Chern, Jiewen Hu, Pengfei Liu

Improving Relation Extraction with Knowledge-attention (D19-1)

Challenge:	Current methods to improve data quality are labor-intensive or prone to factual errors caused by LLM hallucinations.
Approach:	They propose a method which reformats the responses of instruction data into a format that better aligns with pre-established criteria and the collated evidence.
Outcome:	The proposed approach minimizes human annotation, hallucination, and the difficulty in scaling, remaining orthogonal to existing alignment techniques.

Copied to clipboard

Pengfei Li, Kezhi Mao, Xuefeng Yang, Qi Li

Prompt Chaining or Stepwise Prompt? Refinement in Text Summarization (2024.findings-acl)

Challenge:	Existing attention mechanisms are data-driven, but most are data driven.
Approach:	They propose a knowledge-attention encoder which integrates prior knowledge from external lexical resources into deep neural networks for relation extraction task.
Outcome:	The proposed system outperforms existing CNN, RNN, and self-attention based models on a large-scale relation extraction dataset.

Copied to clipboard

Shichao Sun, Ruifeng Yuan, Ziqiang Cao, Wenjie Li, Pengfei Liu

Low-Resource NLP

Challenge:	Large language models (LLMs) can improve summary quality by mirroring a human-like iterative process of critique and refinement starting from the initial draft.
Approach:	They propose to use Prompt Chaining and Stepwise Prompting to perform iterative refinement . they propose to combine the two methods to produce a more favorable outcome .
Outcome:	The proposed methods can improve summary quality by mirroring a human-like iterative process . the results show that the prompt chaining method produces a more favorable outcome .

COVID-19 Claim Radar: A Structured Claim Extraction and Tracking System (2022.acl-demo)

Copied to clipboard

Manling Li, Revanth Gangi Reddy, Ziqi Wang, Yi-shyuan Chiang, Tuan Lai, Pengfei Yu, Zixuan Zhang, Heng Ji

Universal Conditional Masked Language Pre-training for Neural Machine Translation (2022.acl-long)

Challenge:	a new system extracts supporting and refuting claims from COVID-19 related news . the system is publicly available at GitHub and DockerHub, with complete documentation.
Approach:	They propose a COVID-19 Claim Radar system that extracts supporting and refuting claims . the system leverages Wikidata as the hub to consolidate coreferential knowledge elements .
Outcome:	The system extracts supporting and refuting claims from COVID-19 pandemic information . it leverages Wikidata as the hub to merge coreferential knowledge elements .

Copied to clipboard

Pengfei Li, Liangyou Li, Meng Zhang, Minghao Wu, Qun Liu

Multilinguality

Challenge:	Pre-trained sequence-to-sequence models have significantly improved Neural Machine Translation (NMT) this paper demonstrates that pre-training a sequence- to-squence model with a bidirectional decoder can produce notable performance gains for both Autoregressive and Non-autoregressive NMT tasks.
Approach:	They propose a conditional masked language model pre-trained on bilingual and monolingual corpora in many languages.
Outcome:	The proposed model can achieve significant performance improvements on all scenarios from low- to extremely high-resource languages.

WIST: Web-Grounded Iterative Self-Play Tree for Domain-Targeted Reasoning Improvement (2026.acl-long)

Copied to clipboard

Fangyuan Li, Pengfei Li, Shijie Wang, Junqi Gao, Jianxing Liu, Biqing Qi, Yuqiang Li

KG-RAG: Enhancing GUI Agent Decision-Making via Knowledge Graph-Driven Retrieval-Augmented Generation (2025.emnlp-main)

Challenge:	Existing methods for self-improvement of large language models with verifiable rewards (RLVR) can drift over iterations, while corpus-grounded approaches rely on curated data environments.
Approach:	They propose a Web-grounded Iterative Self-play Tree framework for domain-targeted reasoning improvement that learns directly from the open-web without requiring any pre-arranged domain corpus.
Outcome:	The proposed framework outperforms both purely endogenous self-evolution and corpus-grounded self-play baselines and is domain-steerable.

Copied to clipboard

Ziyi Guan, Jason Chun Lok Li, Zhijian Hou, Pingping Zhang, Donglai Xu, Yuzhi Zhao, Mengyang Wu, Jinpeng Chen, Thanh-Toan Nguyen, Pengfei Xian, Wenao Ma, Shengchao Qin, Graziano Chesi, Ngai Wong

SAFETY-J: Evaluating Safety with Critique (2024.findings-emnlp)

Challenge:	Recent advances in GUI agents have limited app-specific knowledge of complex mobile tasks.
Approach:	They propose a Knowledge Graph-driven Retrieval-Augmented Generation framework that transforms fragmented UTGs into structured vector databases for efficient real-time retrieval.
Outcome:	The proposed framework outperforms existing methods in a 75.8% success rate and 84.6% decision accuracy test across mobile apps.

Copied to clipboard

Yixiu Liu, Yuxiang Zheng, Shijie Xia, Jiajun Li, Yi Tu, Chaoling Song, Pengfei Liu

Challenge:	Current methods focus on binary safety classifications and lack detailed critique, limiting their utility for model improvement and user trust.
Approach:	They propose a bilingual generative safety evaluator for English and Chinese with critique-based judgment that utilizes a robust training dataset and augmented query-response pairs to assess safety across various scenarios comprehensively.
Outcome:	The proposed model improves safety evaluations by assessing the quality of critiques with minimal human intervention.

FRoG: Evaluating Fuzzy Reasoning of Generalized Quantifiers in LLMs (2024.emnlp-main)

Copied to clipboard

Yiyuan Li, Shichao Sun, Pengfei Liu

OpenResearcher: Unleashing AI for Accelerated Scientific Research (2024.emnlp-demo)

Challenge:	Existing methods to enhance reasoning do not consistently improve performance in tasks involving fuzzy logic.
Approach:	They propose a benchmark for fuzzy reasoning that incorporates generalized quantifiers.
Outcome:	The proposed benchmark shows that existing methods do not improve on FRoG . strong mathematical reasoning skills are not indicative of success, the authors show .

Copied to clipboard

Yuxiang Zheng, Shichao Sun, Lin Qiu, Dongyu Ru, Cheng Jiayang, Xuefeng Li, Jifan Lin, Binjie Wang, Yun Luo, Renjie Pan, Yang Xu, Qingkai Min, Zizhao Zhang, Yiwen Wang, Wenjie Li, Pengfei Liu

Information Retrieval

Challenge:	Global scientific publications are growing annually by about 4%-5% (Pinedo et al., 2024).
Approach:	They introduce an AI-assisted platform that answers diverse questions from researchers using Retrieval-Augmented Generation (RAG) they develop various tools to understand queries, search from the scientific literature, filter retrieved information, provide accurate and comprehensive answers, and self-refine answers.
Outcome:	OpenResearcher is built on Retrieval-Augmented Generation (RAG) to integrate Large Language Models (LLMs) with up-to-date, domain-specific knowledge.

Modeling Multi-turn Conversation with Deep Utterance Aggregation (C18-1)

Copied to clipboard

Zhuosheng Zhang, Jiangtong Li, Pengfei Zhu, Hai Zhao, Gongshen Liu

Natural Language Interfaces

Challenge:	Existing work on retrieval-based context modeling for multi-turn conversation ignores interactions among previous utterances.
Approach:	They propose retrieval-based response matching for multi-turn conversation . they propose to combine previous utterances into context using a deep utterrance aggregation model .
Outcome:	The proposed model outperforms state-of-the-art methods on three multi-turn conversation benchmarks including an e-commerce dialogue corpus.

Defining a New NLP Playground (2023.findings-emnlp)

Copied to clipboard

Sha Li, Chi Han, Pengfei Yu, Carl Edwards, Manling Li, Xingyao Wang, Yi Fung, Charles Yu, Joel Tetreault, Eduard Hovy, Heng Ji

SegTune: Structured and Fine-Grained Control for Song Generation (2026.acl-long)

Challenge:	Recent explosion of performance of large language models (LLMs) has changed the field more abruptly and seismically than any other shift in the field’s 80 year history.
Approach:	They propose 20+ PhD-dissertation-worthy research directions to define a new NLP playground by combining theoretical analysis, new and challenging problems, learning paradigms and interdisciplinary applications.
Outcome:	The proposed research will cover theoretical analysis, new and challenging problems, learning paradigms and interdisciplinary applications.

Copied to clipboard

Yuejiao Wang, Zihao Ji, Pengfei Cai, Xu Li, Haorui Zheng, Zewen Song, Zhongliang Liu, Chen Zhang, Pengfei Wan

AscendKernelGen: LLM-Driven Kernel Generation for NPUs (2026.findings-acl)

Challenge:	Recent advances in neural song generation have enabled high-quality synthesis from lyrics and global textual prompts.
Approach:	They propose a framework that allows users to specify local musical descriptions aligned to song segments.
Outcome:	The proposed framework outperforms baselines in musicality and controllability.

Copied to clipboard

Xinzi Cao, Jianyang Zhai, Pengfei Li, Zhiheng Hu, Cen Yan, null Mubingxu, Guanghuan Fang, Bin She, Jiayu Li, Yihan Su, Dongyang Tao, Feidiao Yang, Chang-Dong Wang, Yutong Lu, Weicheng Xue, Bin Zhou, Yonghong Tian

Challenge:	Neural Processing Units (NPUs) are critical for AI infrastructure, but their development remains a bottleneck due to vendor-specific Domain-Specific Languages (DSLs).
Approach:	They propose a framework for NPU kernel development that bridges the gap in hardware-specific coding . compiler success on complex Level-2 kernels improves from 0% to 95.5%, they say .
Outcome:	The proposed framework bridges the gap in hardware-specific coding, showing a near-zero success rate on complex kernels.

PEAR: Planner-Executor Agent Robustness Benchmark (2026.findings-eacl)

Copied to clipboard

Shen Dong, Mingxuan Zhang, Pengfei He, Li Ma, Bhavani Thuraisingham, Hui Liu, Yue Xing

Robustness in NLP

Challenge:	Existing studies examine isolated attack surfaces or specific scenarios, leaving a lack of holistic understanding of MAS vulnerabilities.
Approach:	They propose a benchmark to evaluate the utility and vulnerability of planner–executor MAS.
Outcome:	The proposed benchmark evaluates planner–executor MAS on a widely adopted design.

Towards Better Chain-of-Thought: A Reflection on Effectiveness and Faithfulness (2025.findings-acl)

Copied to clipboard

Jiachun Li, Pengfei Cao, Yubo Chen, Jiexin Xu, Huaijun Li, Xiaojian Jiang, Kang Liu, Jun Zhao

Challenge:	Chain-of-thought (CoT) prompting demonstrates varying performance under different reasoning tasks.
Approach:	They propose to recall extra information from the question to enhance CoT generation and evaluate CoTs based on their information gain.
Outcome:	The proposed method improves both the faithfulness and effectiveness of CoT and evaluates it based on their information gain.

M3TQA: Massively Multilingual Multitask Table Question Answering (2026.findings-acl)

Copied to clipboard

Daixin Shu, Jian Yang, Zhenhe Wu, Xianjie Wu, Xianfu Cheng, Guan Xiangyuan, Yanghai Wang, Pengfei Wu, Tingyang Yang, Hualei Zhu, Wei Zhang, Ge Zhang, Jiaheng Liu, Zhoujun Li

Structured Data in NLP

Challenge:	Existing multilingual table benchmarks suffer from geolinguistic imbalance - overrepresenting certain languages and lacking sufficient scale for rigorous cross-lingual analysis.
Approach:	They propose a framework for massively multilingual table question answering that includes tables expanded to 97 languages from Chinese and English sources.
Outcome:	Experiments on state-of-the-art LLMs show that synthetically generated training data significantly boosts performance, especially for low-resource languages.

Retrieval Heads are Dynamic (2026.acl-long)

Copied to clipboard

Yuping Lin, Zitao Li, Yue Xing, Pengfei He, Yingqian Cui, Yaliang Li, Bolin Ding, Jingren Zhou, Jiliang Tang

Information Retrieval

Challenge:	Recent studies have identified "retrieval heads" in Large Language Models responsible for extracting information from input contexts.
Approach:	They propose to examine retrieval heads from a dynamic perspective . they establish that retrieval head activation is highly dynamic and functionally irreplaceable .
Outcome:	The proposed model's hidden state encodes a predictive signal for future retrieval head patterns, indicating an internal planning mechanism.

Towards Dynamic Theory of Mind: Evaluating LLM Adaptation to Temporal Evolution of Human States (2025.acl-long)

Copied to clipboard

Yang Xiao, Jiashuo Wang, Qiancheng Xu, Changhe Song, Chunpu Xu, Yi Cheng, Wenjie Li, Pengfei Liu

Linguistics & Cognitive NLP

Challenge:	Existing benchmarks assess basic Theory of Mind abilities but neglect temporal evolution of mental states in real-world social contexts.
Approach:	They propose a benchmark specifically designed to evaluate Large Language Models' ability to understand and track the temporal progression of mental states across interconnected scenarios.
Outcome:	The proposed benchmarks underperform humans by 44.7% and show that they can model the dynamic nature of human mental states better than existing models.

Complex Event Schema Induction with Knowledge-Enriched Diffusion Model (2023.findings-emnlp)

Copied to clipboard

Yupu Hao, Pengfei Cao, Yubo Chen, Kang Liu, Jiexin Xu, Huaijun Li, Xiaojian Jiang, Jun Zhao

Challenge:	Existing studies on event schema induction have been hindered by errors and data quality issues.
Approach:	They propose a knowledge-enriched discrete diffusion model that distills event scenario knowledge from LLMs.
Outcome:	The proposed model achieves outstanding performance across evaluation metrics.

The Critique of Critique (2024.findings-acl)

Copied to clipboard

Shichao Sun, Junlong Li, Weizhe Yuan, Ruifeng Yuan, Wenjie Li, Pengfei Liu

Challenge:	MetaCritique builds specific quantification criteria to evaluate the quality of critique . a systematic method to evaluate critique is lacking.
Approach:	They propose a critique of critique, termed MetaCritique, which builds specific quantification criteria and aggregates each AIU's judgment for the overall score.
Outcome:	The proposed method can achieve near-human performance across 16 datasets.

Ambiguity Awareness Optimization: Towards Semantic Disambiguation for Direct Preference Optimization (2025.emnlp-main)

Copied to clipboard

Jian Li, Shenglin Yin, Yujia Zhang, Alan Zhao, Xi Chen, Xiaohui Zhou, Pengfei Xu

RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment (2025.findings-acl)

Challenge:	Direct Preference Optimization (DPO) is a widely used reinforcement learning from human feedback (RLHF) method across various domains.
Approach:	They propose an approach that automatically re-weights ambiguous content to reduce ambiguities by calculating semantic similarity from preference pairs.
Outcome:	The proposed approach outperforms state-of-the-art approaches in performance across multiple model scales and widely adopted benchmark datasets.

Copied to clipboard

Zhuoran Jin, Hongbang Yuan, Tianyi Men, Pengfei Cao, Yubo Chen, Jiexin Xu, Huaijun Li, Xiaojian Jiang, Kang Liu, Jun Zhao

Information Retrieval

Challenge:	Existing retrieval augmented language models often overlook effective alignment with human preferences.
Approach:	They propose a benchmark to evaluate RMs in retrieval augmented language models . they incorporate 18 RAG subsets, six retrievers, and 24 RALMs to increase diversity .
Outcome:	The proposed benchmark combines 18 RAG subsets, six retrievers, and 24 RALMs to increase diversity of data sources.

Hierarchical Relation Extraction with Coarse-to-Fine Grained Attention (D18-1)

Copied to clipboard

Xu Han, Pengfei Yu, Zhiyuan Liu, Maosong Sun, Peng Li

From Signal Degradation to Computation Collapse: Uncovering the Two Failure Modes of LLM Quantization (2026.findings-acl)

Challenge:	Existing methods for relation extraction use knowledge graphs to automatically label training data . but, it suffers from the wrong labeling problem because not all sentences containing two entities can express their relations in KGs .
Approach:	They propose a distant supervision approach to automatically label training instances . they integrate hierarchical information of relations into distantly supervised relation extraction .
Outcome:	The proposed model outperforms baseline models on a large-scale dataset.

Copied to clipboard

Chenxi Zhou, Pengfei Cao, Jiang Li, Bohan Yu, Jinyu Ye, Jun Zhao, Kang Liu

Challenge:	Existing research on PTQ spans three primary directions.
Approach:	They conduct a systematic analysis of post-training quantization failures using PTQ . they show that targeted repair can mitigate Signal Degradation but remains ineffective for Computation Collapse .
Outcome:	The proposed method mitigates Signal Degradation but remains ineffective for Computation Collapse.

What Tokens Truly Matter? The Logit Conflation Problem in LLM Sampling (2026.findings-acl)

Copied to clipboard

Pinlong Zhao, Huijun Tang, Pengfei Jiao, Mengyang Li

Task-Stratified Knowledge Scaling Laws for Post-Training Quantized Large Language Models (2026.findings-acl)

Challenge:	Existing methods for large language models filter tokens based on logit magnitudes or derived statistics, under the implicit assumption that high-logit tokens are desirable.
Approach:	They propose to isolate the Logit Conflation Problem by using attention-weighted attribution to extract prompt-relevance from token logits.
Outcome:	The proposed method improves on LLaMA-3 and is training-free and low latency.

Copied to clipboard

Chenxi Zhou, Pengfei Cao, Jiang Li, Bohan Yu, Jinyu Ye, Jun Zhao, Kang Liu

The Law of Knowledge Overshadowing: Towards Understanding, Predicting and Preventing LLM Hallucination (2025.findings-acl)

Challenge:	Existing scaling laws focus on general performance, overlooking crucial fine-grained factors and how quantization differentially impacts diverse knowledge capabilities.
Approach:	They propose a framework that unifies model size, bit-width, and fine-grained factors into memorization, application, and reasoning.
Outcome:	The proposed framework shows strong fit and cross-architecture consistency on 293 different PTQ configurations.

Copied to clipboard

Yuji Zhang, Sha Li, Cheng Qian, Jiateng Liu, Pengfei Yu, Chi Han, Yi R. Fung, Kathleen McKeown, ChengXiang Zhai, Manling Li, Heng Ji

Lightweight Haar Wavelet Subband Pruning for LLMs (2026.findings-acl)

Challenge:	Hallucination is a persistent challenge in large language models where even with rigorous quality control, models often generate distorted facts.
Approach:	They propose a new framework to quantify factual hallucinations by modeling knowledge overshadowing.
Outcome:	The proposed framework improves model factuality on Overshadow (27.9%), MemoTrap (13.1%) and NQ-Swap (18.3%).

Copied to clipboard

Jiang Li, Pengfei Cao, Chenxi Zhou, Tian Lan, Xiangdong Su, Kang Liu, Jun Zhao, Guanglai Gao

Challenge:	Large language models (LLMs) have impressive performance but require computational and memory resources.
Approach:	They propose a post-training framework that uses a Haar wavelet transform to prune weights.
Outcome:	The proposed pruning framework reduces pruning time and computational costs by removing less important weights while preserving model architecture.

Self-supervised Preference Optimization: Enhance Your Language Model with Preference Degree Awareness (2024.findings-emnlp)

Copied to clipboard

Jian Li, Haojing Huang, Yujia Zhang, Pengfei Xu, Xi Chen, Rui Song, Lida Shi, Jingwen Wang, Hao Xu

AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts (2026.acl-long)

Challenge:	Recent studies have focused on replacing the reward model in Reinforcement Learning with Human Feedback (RLHF) methods for Large Language Models (LLMs).
Approach:	They propose a self-supervised preference optimization framework that replaces the reward model with a preference loss and alignment loss to improve LLMs' ability to understand human preferences.
Outcome:	The proposed framework can be integrated with existing preference optimization methods and significantly boost their performance.

Copied to clipboard

Keyu Li, Junhao Shi, Yang Xiao, Mohan Jiang, Jie Sun, Yunze Wu, Dayuan Fu, Shijie Xia, Xiaojie Cai, Tianze Xu, Weiye Si, Wenjie Li, Dequan Wang, Pengfei Liu

Challenge:	Existing benchmarks focus on single agentic capability, failing to capture long-horizon real-world scenarios.
Approach:	They propose a benchmark that evaluates 6 agentic capabilities across 32 real-world scenarios.
Outcome:	Experiments show that closed-source models outperform open-source model (48.4% vs 32.1%) integrating models with advanced scaffolds to form autonomous agents is a paradigm shift.

AIMMerging: Adaptive Iterative Model Merging Using Training Trajectories for Language Model Continual Learning (2025.emnlp-main)

Copied to clipboard

Yujie Feng, Jian Li, Xiaoyu Dong, Pengfei Xu, Xiaohui Zhou, Yujia Zhang, Zexin Lu, Yasha Wang, Alan Zhao, Xu Chu, Xiao-Ming Wu

AgentsCourt: Building Judicial Decision-Making Agents with Court Debate Simulation and Legal Knowledge Augmentation (2024.findings-emnlp)

Challenge:	Recent model merging-based methods struggle to effectively manage the trade-off between learning new knowledge and preventing catastrophic forgetting.
Approach:	They propose a model merging framework that utilizes learning and forgetting signals from the training trajectory to dynamically monitor the model’s training status.
Outcome:	The proposed framework achieves significant performance improvements over existing state-of-the-art methods on three CL benchmarks with various model sizes (from 770M to 13B).

Copied to clipboard

Zhitao He, Pengfei Cao, Chenhao Wang, Zhuoran Jin, Yubo Chen, Jiexin Xu, Huaijun Li, Kang Liu, Jun Zhao

LASS: A Novel and Economical Data Augmentation Framework Based on Language Models for Debiasing Opinion Summarization (2025.coling-main)

Challenge:	Recent advances in deep learning have significantly impacted the legal domain.
Approach:	They propose a multi-agent framework for judicial decision-making that simulates the court trial process . they propose 420 Chinese judgment documents to support their framework and build a large-scale legal knowledge base .
Outcome:	The proposed framework outperforms existing methods in various aspects, especially in generating legal articles.

Copied to clipboard

Yanyue Zhang, Pengfei Li, Yilong Lai, Yulan He, Deyu Zhou

Responsible & Trustworthy NLP

Challenge:	Existing methods to generate negative summaries are expensive and lack the capacity to generate large data sets.
Approach:	They propose a data augmentation framework based on LArge and Small language models for debiaSing opinion summarization that generates a small number of synthesized negative reviews by rewriting the positive text via a large language model.
Outcome:	The proposed framework can generate large numbers of negative reviews by rewriting the positive text using a large language model and training a disentangle reconstruction model based on the generated data.

Opinions Are Not Always Positive: Debiasing Opinion Summarization with Model-Specific and Model-Agnostic Methods (2024.lrec-main)

Copied to clipboard

Yanyue Zhang, Yilong Lai, Zhenglin Wang, Pengfei Li, Deyu Zhou, Yulan He

Sentiment Analysis

Challenge:	Existing opinion summarization frameworks are reluctant to generate negative summaries given input of negative opinions.
Approach:	They propose to disentangle input into sentiment-relevant and sentiment-irrelevant components through adversarial loss.
Outcome:	The proposed approaches reduce sentiment bias in the existing opinion summarization dataset . the proposed approaches generate better summaries with a more balanced emotional polarity distribution .

Lingke: a Fine-grained Multi-turn Chatbot for Customer Service (C18-2)

Copied to clipboard

Pengfei Zhu, Zhuosheng Zhang, Jiangtong Li, Yafang Huang, Hai Zhao

Natural Language Interfaces

Challenge:	e-commerce chatbots usually need a mass of human dialogue data to train, but for multi-turn conversations, the performance is poor.
Approach:	They propose an information retrieval augmented multi-turn chatbot which can answer questions based on unstructured documents and deal with multi-turned conversations.
Outcome:	The proposed solution outperforms all other models in multi-turn conversations and can learn from conversation records.

Leros: Learning Explicit Reasoning on Synthesized Data for Commonsense Question Answering (2024.lrec-main)

Copied to clipboard

Chenhao Wang, Pengfei Cao, Jiachun Li, Yubo Chen, Kang Liu, Xiaojian Jiang, Jiexin Xu, Li Qiuxia, Jun Zhao

Benchmarking Temporal Reasoning and Alignment Across Chinese Dynasties (2026.eacl-short)

Challenge:	Recent work shows large language models can generate useful rationales for commonsense question answering (CQA) however, the cost of deployment and further tuning is relatively expensive for the large models.
Approach:	They propose a framework that leverages both knowledge graphs and large language models to synthesize rationale-augmented CQA data.
Outcome:	The proposed model can generate useful rationales on unseen CQA benchmarks.

Copied to clipboard

Zhenglin Wang, Jialong Wu, Pengfei Li, Yong Jiang, Deyu Zhou