Papers by Shengjie Wang
An End-to-End Submodular Framework for Data-Efficient In-Context Learning (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent advances in natural language tasks leverage the emergent In-Context Learning ability of pretrained Large Language Models (LLMs). |
| Approach: | They propose a framework for exemplar selection for in-context learning that uses a pool-based active learning approach to select Diverse and informative exemplars from the target tasks’ unlabeled pool. |
| Outcome: | The proposed framework outperforms existing methods for data annotation and similarity-based methods for test query-specific exemplar retrieval on 7 different NLP datasets and 5 LLMs of varying complexities. |
Towards Hierarchical Multi-Step Reward Models for Enhanced Reasoning in Large Language Models (2026.findings-acl)
Copied to clipboard
Teng Wang, Jiang Zhangyi, Zhenqi He, Hailei Gong, Shenyang Tong, Wenhan Yang, Zeyu Li, Yanan Zheng, Zifan He, Zewen Ye, Shengjie Ma, Jianping Zhang
| Challenge: | Existing Process Reward Models (PRMs) are vulnerable to reward hacking and require expensive, large-scale annotation of reasoning steps. |
| Approach: | They propose a reward model approach which evaluates both individual and consecutive reasoning steps from fine-grained and coarse-grounded level. |
| Outcome: | Empirical results show that the proposed model performs better than existing PRMs and is more robust than existing models. |
BadAgent: Inserting and Activating Backdoor Attacks in LLM Agents (2024.acl-long)
Copied to clipboard
| Challenge: | Xi et al., 2023) show that LLM-based intelligent agents are vulnerable to backdoor attacks. |
| Approach: | They propose to build LLM-based intelligent agents using trained LLMs and fine-tune them on data for the agent task. |
| Outcome: | The proposed methods are robust even after fine-tuning on trusted data. |
CLEAR: Can Language Models Really Understand Causal Graphs? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing language models lack a conceptual framework for understanding causal graphs, but there is still potential for improvement. |
| Approach: | They develop a framework to define causal graph understanding by assessing language models’ behaviors through four practical criteria derived from diverse disciplines. |
| Outcome: | The proposed framework defines three complexity levels and encompasses 20 causal graph-based tasks across 20 different levels. |
JudgeAgent: Beyond Static Benchmarks for Knowledge-Driven and Dynamic LLM Evaluation (2026.findings-acl)
Copied to clipboard
Zhichao Shi, Xuhui Jiang, Chengjin Xu, Cangli Yao, Shengjie Ma, Yinghan Shen, Zixuan Li, Jian Guo, Yuanzhuo Wang
| Challenge: | Current evaluation methods for large language models rely on static benchmarks . limited knowledge coverage and fixed difficulties hinder the targeted optimizations resulting in superficial evaluations of LLMs - a problem that has been addressed by JudgeAgent . |
| Approach: | They propose a knowledge-driven and dynamic evaluation framework for large language models . judgeAgent leverages LLM agents equipped with context graphs to traverse knowledge structures . |
| Outcome: | The proposed framework can achieve comprehensive evaluations and facilitate effective model iterations. |
STAND-Guard: A Small Task-Adaptive Content Moderation Model (2025.coling-industry)
Copied to clipboard
| Challenge: | Content moderation is important for developing welcoming online platforms and responsible large language models. |
| Approach: | They propose a small task-adaptive coNtent moDeration model that can be easily adapted to new or customized content moderation tasks without extensive model tuning. |
| Outcome: | The proposed model is comparable to GPT-3.5-Turbo on unseen English binary classification tasks. |
AdvancedIF: Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following (2026.acl-long)
Copied to clipboard
Yun He, Wenzhe Li, Hejia Zhang, Songlin Li, Karishma Mandyam, Sopan Khosla, Yuanhao Xiong, Nanshu Wang, Xiaoliang Peng, Beibin Li, Shengjie Bi, Shishir G Patil, Qi Qi, Shengyu Feng, Julian Katz-Samuels, Richard Yuanzhe Pang, Sujan Kumar Gonugondla, Hunter Lang, Yue Yu, Yundi Qian, Maryam Fazel-Zarandi, Licheng Yu, Amine Benhalloum, Hany Hassan Awadalla, Manaal Faruqui
| Challenge: | Recent advances in large language models (LLMs) have shown impressive performance on a range of tasks, yet advanced instruction following (IF) remains a significant challenge. |
| Approach: | They propose a benchmark that features over 1,600 prompts and expert-curated rubrics that assess LLMs’ ability to follow complex, multi-turn, and system-level instructions. |
| Outcome: | The proposed framework improves instruction-following abilities of large language models, achieving a 6.7% gain on AdvancedIF and strong results on public benchmarks. |
How Numerical Precision Affects Arithmetical Reasoning Capabilities of LLMs (2025.findings-acl)
Copied to clipboard
Guhao Feng, Kai Yang, Yuntian Gu, Xinyue Ai, Shengjie Luo, Jiacheng Sun, Di He, Zhenguo Li, Liwei Wang
| Challenge: | Despite the success of transformer-based large language models, understanding and enhancing their mathematical capabilities remains a significant challenge. |
| Approach: | They propose to use numerical precision as a key factor that influences LLMs' effectiveness in arithmetical tasks to determine their effectiveness. |
| Outcome: | The proposed models perform better in arithmetic tasks than transformer-based models with standard numerical precision. |
AdapThink: Adaptive Thinking Preferences for Reasoning Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent research has highlighted a significant inefficiency associated with the slow thinking paradigm . models often overthink simple tasks while underthinking complex challenges . |
| Approach: | They propose a framework for adaptive reasoning preference control that dynamically adjusts reflection preferences based on group-level distributional statistics of reasoning length and reflection intensity. |
| Outcome: | The proposed framework reduces average response length by 17.1%-21.4% while improving performance by 6.12-6.59 points under 32K token budgets. |