Papers by Weipeng Zhang
FB-Bench: A Fine-Grained Multi-Task Benchmark for Evaluating LLMs’ Responsiveness to Human Feedback (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing research focuses on benchmarking LLMs in single-turn dialogues, neglecting the nuanced nature of human feedback within real-world usage scenarios. |
| Approach: | They propose a fine-grained, multi-task benchmark designed to evaluate LLMs’ responsiveness to human feedback under real-world usage scenarios in Chinese. |
| Outcome: | The proposed benchmarks show that human feedback can significantly impact LLMs’ responsiveness in real-world usage scenarios. |
Unveiling the Flaws: Exploring Imperfections in Synthetic Data and Mitigation Strategies for Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | acquiring large amounts of high-quality data can be challenging due to data scarcity, privacy concerns, and high costs. |
| Approach: | They propose a method which reverses instruction-following issues caused by uniform format of synthetic data and proposes unlearning techniques to mitigate these flaws. |
| Outcome: | The proposed method reverses instruction-following issues caused by pattern overfitting without compromising performance on benchmarks at relatively low cost. |
From Paraphrasing to Semantic Parsing: Unsupervised Semantic Parsing via Synchronous Semantic Decoding (2021.acl-long)
Copied to clipboard
Shan Wu, Bo Chen, Chunlei Xin, Xianpei Han, Le Sun, Weipeng Zhang, Jiansong Chen, Fan Yang, Xunliang Cai
| Challenge: | Experimental results show that Synchronous Semantic Decoding (SSD) can achieve state-of-the-art unsupervised semantic parsing performance on multiple datasets. |
| Approach: | They propose an unsupervised method which solves the semantic gap and the structure gap by leveraging paraphrasing and grammar-constrained decoding. |
| Outcome: | The proposed method can solve the semantic gap and structure gap on multiple datasets. |
False Friends in the Shell: Unveiling the Emoticon Semantic Confusion in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Emoticons are widely used in digital communication to convey affective intent, yet their safety implications for Large Language Models (LLMs) remain largely unexplored. |
| Approach: | They propose to use ASCII-based emoticons to perform unintended actions in large language models (LLMs) This vulnerability is pervasive, with an average confusion ratio exceeding 38%, and 90% of confused responses yield 'silent failures' authors call on the community to recognize this emerging vulnerability and develop effective mitigation methods to uphold the safety and reliability of human-LLM interactions. |
| Outcome: | The proposed framework exploits emoticon semantic confusion in six LLMs and demonstrates that existing prompt-based mitigations are ineffective. |
PaTaRM: Bridging Pairwise and Pointwise Signals via Preference-Aware Task-Adaptive Reward Modeling (2026.acl-long)
Copied to clipboard
| Challenge: | Existing reward models lack generative and reasoning capabilities, resulting in poor performance. |
| Approach: | They propose a reward-aware task-adaptive reward model that enables pointwise training using readily available pairwise data via a novel Preference-Aware Reward mechanism. |
| Outcome: | The proposed reward model achieves an average relative improvement of 8.7% over the base models on RewardBench and RMBench. |
ShortGPT: Layers in Large Language Models are More Redundant Than You Expect (2025.findings-acl)
Copied to clipboard
Xin Men, Mingyu Xu, Qingyu Zhang, Qianhao Yuan, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, Weipeng Chen
| Challenge: | Recent studies have identified significant redundancy in large language models . quantization and pruning are two methods that reduce computational resources . |
| Approach: | They propose simple pruning methods that prune redundant layers based on their BI scores. |
| Outcome: | The proposed pruning methods demonstrate superior performance over previous pruning methods. |
CFBench: A Comprehensive Constraints-Following Benchmark for LLMs (2025.acl-long)
Copied to clipboard
Tao Zhang, ChengLIn Zhu, Yanjun Shen, Wenjing Luo, Yan Zhang, Hao Liang, Tao Zhang, Fan Yang, Mingan Lin, Yujing Qiao, Weipeng Chen, Bin Cui, Wentao Zhang, Zenan Zhou
| Challenge: | Existing evaluations of Large Language Models (LLMs) focus on fragmented constraints or narrow scenarios, but they overlook the comprehensiveness and authenticity of constraints from the user’s perspective. |
| Approach: | They propose a Chinese Comprehensive Constraints Following Benchmark for LLMs that compiles constraints from real-world instructions and constructs a systematic framework for constraint types. |
| Outcome: | The proposed framework integrates multi-dimensional assessment criteria with requirement prioritization, covering various perspectives of constraints, instructions, and requirement fulfillment. |
Harmonizing Dense and Sparse Signals in Multi-turn RL: Dual-Horizon Credit Assignment for Industrial Sales Agents (2026.acl-industry)
Copied to clipboard
| Challenge: | Large language models for industrial sales require balancing long-term commercial objectives with immediate linguistic constraints such as fluency and compliance. |
| Approach: | They propose a framework that disentangles optimization across time scales by normalizing advantages from turn-level and session-level rewards before fusion. |
| Outcome: | The proposed framework outperforms the state-of-the-art GRPO model in conversion rate and identity detection rate. |
The Invisible Hand: Unveiling Provider Bias in Large Language Models for Code Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have emerged as the new recommendation engines, surpassing traditional methods in both capability and scope, particularly in code generation. |
| Approach: | They propose to use a dataset to investigate a new type of bias in Large Language Models for code generation, provider bias, to determine whether the model favors specific providers. |
| Outcome: | The proposed model favors services from Google and Amazon, but without explicit directives, and can modify input code to incorporate their preferred providers without user requests. |