Papers by Huan Wang

ChatMap: Mining Human Thought Processes for Customer Service Chatbots via Multi-Agent Collaboration (2025.findings-acl)

Challenge:	Existing text sanitization mechanisms provide low utility, as cursed by the high-dimensional text representation.
Approach:	They propose to use sanitized texts to samaritize training data . they propose to retrain and fine-tune the senitization-aware language model .
Outcome:	The proposed approach enables privacypreserving natural language processing over the BERT language model with promising utility.

Copied to clipboard

Xinyi Jiang, Tianyi Hu, Yuheng Qin, Guoming Wang, Zhou Huan, Kehan Chen, Gang Huang, Rongxing Lu, Siliang Tang

ActionStudio: A Lightweight Framework for Data and Training of Large Action Models (2025.emnlp-main)

Challenge:	Existing methods for enhancing dialogue performance rely on summarizing behavior . e-commerce chatbots need to align their dialogue strategies with human behavior to achieve coherent, human-like conversations with customers.
Approach:	They propose a method to extract core patterns from dialogue data and integrate them into models by mining service thought processes using a multi-agent aPproach.
Outcome:	The proposed method outperforms manual methods and outperfies baselines on Taobao in China.

Copied to clipboard

Jianguo Zhang, Thai Quoc Hoang, Ming Zhu, Zuxin Liu, Shiyu Wang, Tulika Manoj Awalgaonkar, Akshara Prabhakar, Haolin Chen, Weiran Yao, Zhiwei Liu, Juntao Tan, Juan Carlos Niebles, Shelby Heinecke, Huan Wang, Silvio Savarese, Caiming Xiong

Challenge:	Existing infrastructure for efficient agentic data processing and model training remains underdeveloped.
Approach:	They propose a lightweight and extensible data and training framework for large action models . they propose to unify diverse agent trajectories using Unified Format 2.0 .
Outcome:	The proposed framework shows 9 higher throughput than existing frameworks and performs well across public and realistic agent benchmarks.

ReportLogic: Evaluating Logical Quality in Deep Research Reports (2026.acl-long)

Copied to clipboard

Jujia Zhao, Zhaoxin Huan, Zihan Wang, Xiaolu Zhang, Jun Zhou, Suzan Verberne, Zhaochun Ren

Robustness in NLP

Challenge:	Existing evaluation frameworks that evaluate large language models for Deep Research largely ignore this requirement.
Approach:	They propose a benchmark that quantifies report-level logical quality through a reader-centric lens of auditability.
Outcome:	The proposed model quantifies logical quality through a reader-centric lens of auditability.

Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding (2024.acl-long)

Copied to clipboard

Jun Zhang, Jue Wang, Huan Li, Lidan Shou, Ke Chen, Gang Chen, Sharad Mehrotra

Iteratively Prompt Pre-trained Language Models for Chain of Thought (2022.emnlp-main)

Challenge:	Existing methods for accelerating Large Language Models have been criticized for their inference costs and inefficient decoding.
Approach:	They propose a self-speculative decoding approach for accelerating Large Language Models without an auxiliary model.
Outcome:	The proposed method achieves a speedup of up to 1.99 with no additional neural network training and no extra memory footprint.

Copied to clipboard

Boshi Wang, Xiang Deng, Huan Sun

PersonaBench: Evaluating AI Models on Understanding Personal Information through Accessing (Synthetic) Private User Data (2025.findings-acl)

Challenge:	Pre-trained language models (PLMs) internalize a great amount of knowledge, but have been shown incapable of recalling this knowledge to solve complex & multi-step reasoning tasks.
Approach:	They propose an iterative prompting framework which progressively elicits relevant knowledge from PLMs for multi-step inference.
Outcome:	The proposed prompting framework outperforms existing prompting methods on three datasets involving multi-step reasoning.

Copied to clipboard

Juntao Tan, Liangwei Yang, Zuxin Liu, Zhiwei Liu, Rithesh R N, Tulika Manoj Awalgaonkar, Jianguo Zhang, Weiran Yao, Ming Zhu, Shirley Kokane, Silvio Savarese, Huan Wang, Caiming Xiong, Shelby Heinecke

BatchMixup: Improving Training by Interpolating Hidden States of the Entire Mini-batch (2021.findings-acl)

Challenge:	Existing research lacks direct access to such data, making benchmarking difficult due to privacy concerns.
Approach:	They propose a synthetic data pipeline that generates realistic user profiles and private documents and a benchmark to evaluate models' ability to understand personal information.
Outcome:	The proposed pipeline generates realistic user profiles and private documents, enabling PersonaBench, a benchmark for evaluating models’ ability to understand personal information.

Copied to clipboard

Wenpeng Yin, Huan Wang, Jin Qu, Caiming Xiong

ETR: Entropy Trend Reward for Efficient Chain-of-Thought Reasoning (2026.acl-long)

Challenge:	a data augmentation technique is used to augment data, but it has two drawbacks.
Approach:	They propose a new mixup paradigm that generates new points scattered throughout the whole mini-batch.
Outcome:	The proposed model improves the performance of NLP tasks while using different ratios of training data.

Copied to clipboard

Xuan Xiong, Huan Liu, Li Gu, Zhixiang Chi, Yue Qiu, Yuanhao YU, Yang Wang

InsBank: Evolving Instruction Subset for Ongoing Alignment (2025.findings-emnlp)

Challenge:	Existing methods to shorten CoTs use length penalties or global entropy reduction . Existing approaches to CoT reasoning have significant practical drawbacks .
Approach:	They propose a method that shortens CoTs by length penalties or global entropy reduction . they integrate ETR into Group Relative Policy Optimization and evaluate it .
Outcome:	The proposed objective improves accuracy–efficiency trade-off by +9.9% while reducing CoT length by 67% across four benchmarks.

Copied to clipboard

Jiayi Shi, Yiwei Li, Shaoxiong Feng, Peiwen Yuan, Xinglin Wang, Yueqi Zhang, Chuyi Tan, Boyuan Pan, Huan Ren, Yao Hu, Kan Li

Green & Sustainable NLP

Challenge:	Recent studies emphasize that quality and diversity of instruction data are more crucial than quantity, highlighting the need to select diverse, high-quality subsets to reduce training costs.
Approach:	They propose to use a continuously updated repository to integrate the latest valuable instruction data with a progressive evolution framework to evolve InsBank over time.
Outcome:	The proposed framework outperforms baselines in InsBank evolution and extracts budget-specific subsets.

Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters (2023.acl-long)

Copied to clipboard

Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, Huan Sun

Low-Resource NLP

Challenge:	Chain-of-Thought (CoT) prompting can dramatically improve the multi-step reasoning abilities of large language models (LLMs).
Approach:	They propose to use Chain-of-Thought (CoT) prompting to encourage the LLM to generate intermediate rationales for solving a problem by providing a series of reasoning steps in the demonstrations.
Outcome:	The proposed model can generate coherent lines of reasoning even with invalid demonstrations while still generating coherent lines during inference.

In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents (2025.acl-long)

Copied to clipboard

Zhen Tan, Jun Yan, I-Hung Hsu, Rujun Han, Zifeng Wang, Long Le, Yiwen Song, Yanfei Chen, Hamid Palangi, George Lee, Anand Rajan Iyer, Tianlong Chen, Huan Liu, Chen-Yu Lee, Tomas Pfister

Natural Language Interfaces

Challenge:	Existing approaches to long-term dialogue memory management fail to capture the natural semantic structure of conversations, leading to fragmented and incomplete representations.
Approach:	They propose a mechanism that integrates forward- and backward-looking reflections into a personalized memory bank for effective future retrieval.
Outcome:	The proposed mechanism outperforms state-of-the-art benchmarks on a long-term dialogue memory model.

An Emotional Comfort Framework for Improving User Satisfaction in E-Commerce Customer Service Chatbots (2021.naacl-industry)

Copied to clipboard

Shuangyong Song, Chao Wang, Haiqing Chen, Huan Chen

Natural Language Interfaces

Challenge:	E-commerce has grown rapidly over the last several years, and chatbots for intelligent customer service are simultaneously drawing attention.
Approach:	They propose a framework to obtain proper answer to customers’ emotional questions using emotion classification model and text matching.
Outcome:	The proposed framework is very promising on real online systems.

WR-One2Set: Towards Well-Calibrated Keyphrase Generation (2022.emnlp-main)

Copied to clipboard

Binbin Xie, Xiangpeng Wei, Baosong Yang, Huan Lin, Jun Xie, Xiaoli Wang, Min Zhang, Jinsong Su

Text Generation

Challenge:	Experimental results show that keyphrase generation has serious calibration errors . ONE2SET generates short phrases summarizing an input document .
Approach:	They propose a paradigm for keyphrase generation that generates short phrases summarizing an input document.
Outcome:	The proposed model over-estimates tokens and makes it well-calibrated on common datasets.

AlphaOne: Reasoning Models Thinking Slow and Fast at Test Time (2025.emnlp-main)

Copied to clipboard

Junyu Zhang, Runpei Dong, Han Wang, Xuying Ning, Haoran Geng, Peihao Li, Xialin He, Yutong Bai, Jitendra Malik, Saurabh Gupta, Huan Zhang

Unimodal and Crossmodal Refinement Network for Multimodal Sequence Fusion (2021.emnlp-main)

Challenge:	Existing monotonic scaling methods for large reasoning models are not reliable.
Approach:	They propose a universal framework for modulating reasoning progress in large reasoning models at test time.
Outcome:	The proposed framework unifies and generalizes existing monotonic scaling methods and enables flexible and dense slow-to-fast reasoning modulation.

Copied to clipboard

Xiaobao Guo, Adams Kong, Huan Zhou, Xianfeng Wang, Min Wang

Robustness in NLP

Challenge:	Existing approaches to modulate one modal feature to another are lacking in multimodal representation learning.
Approach:	They propose to use unimodal and crossmodal refinement networks to enhance uni and cross-modal representations by iterative updating of distributions with transformer-based attention layers to refine modality-specific learning.
Outcome:	The proposed network outperforms state-of-the-art techniques on MOSI and MOSEI datasets.

DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AI (2024.findings-eacl)

Copied to clipboard

Jianguo Zhang, Kun Qian, Zhiwei Liu, Shelby Heinecke, Rui Meng, Ye Liu, Zhou Yu, Huan Wang, Silvio Savarese, Caiming Xiong

Be More with Less: Hypergraph Attention Networks for Inductive Text Classification (2020.emnlp-main)

Challenge:	DialogStudio is the largest and most diverse collection of dialogue datasets . existing datasets lack diversity and comprehensiveness, authors say .
Approach:	They introduce DialogStudio: the largest and most diverse collection of dialogue datasets . DialogStuio aggregates more than 80 diverse dialogue dataset .
Outcome:	a new dataset is created to improve the quality and diversity of dialogue datasets . DialogStudio is the largest and most diverse collection of dialogue data .

Copied to clipboard

Kaize Ding, Jianling Wang, Jundong Li, Dingcheng Li, Huan Liu

Information Extraction & Text Mining

Challenge:	Text classification is a critical research topic with broad applications in natural language processing. graph neural networks (GNNs) have received increasing attention but their performance is jeopardized in practice.
Approach:	They propose a model which captures long-distance interactions between words and a graph-based model which can be used to perform text classification.
Outcome:	The proposed model can achieve more expressive power with less computational consumption on the text classification task.

Internalizing Multi-Agent Reasoning for Accurate and Efficient LLM-based Recommendation (2026.findings-acl)

Copied to clipboard

Yang Wu, Haoze Wang, Qian Li, Jun Zhang, Huan Yu, Jie Jiang

Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens (2026.findings-acl)

Challenge:	Large Language Models (LLMs) are reshaping recommender systems by leveraging extensive world knowledge and semantic reasoning to interpret user intent.
Approach:	They propose a single-agent Trajectory-Aligned Recommender to integrate reasoning capabilities into a model by a multi-agend teacher system.
Outcome:	The proposed model surpasses its teacher by 8.7% to 39.5% while eliminating iterative latency.

Copied to clipboard

Chengshuai Zhao, Zhen Tan, Pingchuan Ma, Dawei Li, Bohan Jiang, Yancheng Wang, Yingzhen Yang, Huan Liu

Reasoning

Challenge:	Chain-of-Thought (CoT) prompting has been shown to be effective in eliciting structured reasoning from large language models (LLMs).
Approach:	They propose a data distribution lens to understand when and why CoT reasoning fails . they propose 'data-based' training that trains LLMs from scratch .
Outcome:	The proposed model enables models to generate reasoning trajectories that approximate those observed during training.

SALT: Step-level Advantage Assignment for Long-horizon Agents via Trajectory Graph (2026.findings-eacl)

Copied to clipboard

Jiazheng Li, Yawei Wang, Qiaojing Yan, Yijun Tian, Zhichao Xu, Huan Song, Panpan Xu, Lin Lee Cheong

Structured Data in NLP

Challenge:	Large Language Models (LLMs) have demonstrated remarkable capabilities, but their application to complex, multi-step, and long-horizon tasks remains challenging.
Approach:	They propose a framework that provides a finer-grained advantage assignment derived solely from outcome rewards.
Outcome:	The proposed framework provides a finer-grained advantage assignment, derived solely from outcome rewards.

Unsupervised Paraphrasing with Pretrained Language Models (2021.emnlp-main)

Copied to clipboard

Tong Niu, Semih Yavuz, Yingbo Zhou, Nitish Shirish Keskar, Huan Wang, Caiming Xiong

MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark (2025.acl-long)

Challenge:	Paraphrase generation has benefited from recent advances in the design of training objectives and model architectures, but previous studies focused on supervised methods that require a large amount of labeled data that is costly to collect.
Approach:	They propose a transfer learning approach that enables pre-trained language models to generate high-quality paraphrases in an unsupervised setting.
Outcome:	The proposed model performs state-of-the-art on the Quora Question Pair and ParaNMT datasets and is robust to domain shift between the two datasets.

Copied to clipboard

Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, Yu Su, Wenhu Chen, Graham Neubig

Sibyl: Empowering Empathetic Dialogue Generation in Large Language Models via Sensible and Visionary Commonsense Inference (2025.coling-main)

Challenge:	Recent advances in multimodal large language models have led to progress in tackling complex reasoning tasks that combine textual and visual information.
Approach:	They introduce a robust version of the Massive Multi-discipline Multimodal Understanding and Reasoning (MMMU) benchmark.
Outcome:	The proposed model performs lower on MMMU-Pro than on the previous benchmark, ranging from 16.8% to 26.9%.

Copied to clipboard

Lanrui Wang, Jiangnan Li, Chenxu Yang, Zheng Lin, Hongyin Tang, Huan Liu, Yanan Cao, Jingang Wang, Weiping Wang

Double: Breaking the Acceleration Limit via Double Retrieval Speculative Parallelism (2026.acl-long)

Challenge:	Recent studies have focused on integrating commonsense knowledge into chatbots to enhance their ability to understand and generate dialogue responses.
Approach:	They propose a framework that integrates commonsense knowledge into chatbots to enable them to elicit more empathetic responses.
Outcome:	The proposed framework enables LLMs to uncover the implicit requirements of the conversation, aiming to elicit more empathetic responses.

Copied to clipboard

Yuhao Shen, Tianyu Liu, Junyi Shen, Jinyang Wu, Quan Kong, Huan Li, Cong Wang

Information Retrieval

Challenge:	Parallel Speculative Decoding (PSD) has limitations due to speedup limits and high computational waste . a novel synchronous mechanism solves the Retrieval Precision-Efficiency Dilemma .
Approach:	They propose a framework that combines a draft-verification-based approach with a synchronous mechanism to solve the Retrieval Precision-Efficiency Dilemma.
Outcome:	The proposed framework breaks speedup limits for Speculative Decoding by overlapping draft generation with verification.

Estimating the Black-box LLM Uncertainty with Distribution-Aligned Adversarial Distillation (2026.acl-long)

Copied to clipboard

Huizi Cui, Huan Ma, Qilin Wang, Yuhang Gao, Changqing Zhang

How Trustworthy are Open-Source LLMs? An Assessment under Malicious Demonstrations Shows their Vulnerabilities (2024.naacl-long)

Challenge:	Existing uncertainty quantification methods depend on computationally expensive multiple sampling or internal parameters, which prevents real-time estimation and fails to capture information implicit in the black-box reasoning process.
Approach:	They propose a distribution-aligned adjudication architecture to guide a lightweight proxy model to learn the high-quality regions of the output distribution of the black-box LLM.
Outcome:	Extensive experiments show that a proxy model even with 1% of the target LLM’s size can achieve reliable uncertainty quantification.

Copied to clipboard

Lingbo Mo, Boshi Wang, Muhao Chen, Huan Sun

MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application (2026.acl-long)

Challenge:	Rapid progress in open-source Large Language Models (LLMs) is driving AI development, but lacks sufficient trustworthiness to detect and mitigate adversarial demonstrations.
Approach:	They propose an extended Chain of Utterances-based (CoU) prompting strategy to attack open-source LLMs.
Outcome:	The proposed attack strategy is based on malicious demonstrations and toxicity tests on open-source models.

Copied to clipboard

Xueqing Peng, Lingfei Qian, Yan Wang, Ruoyu Xiang, Yueru He, Yang Ren, Mingyang Jiang, Vincent Jim Zhang, Yuqing Guo, Jeff Zhao, Huan He, Yi Han, Yun Feng, Yuechen Jiang, Yupeng Cao, Haohang Li, Yangyang Yu, Xiaoyu Wang, Penglei Gao, Shengyuan Lin, Keyi Wang, Shanshan Yang, Yilun Zhao, Zhiwei Liu, Peng Lu, Jerry Huang, Suyuchen Wang, Triantafillos Papadopoulos, Polydoros Giannouris, Efstathia Soufleri, Nuo Chen, Zhiyang Deng, Heming Fu, Yijia Zhao, Mingquan Lin, Meikang Qiu, Kaleb E Smith, Arman Cohan, Xiao-Yang Liu, Jimin Huang, Guojun Xiong, Alejandro Lopez-Lira, Xi Chen, Junichi Tsujii, Jian-Yun Nie, Sophia Ananiadou, Qianqian Xie

Persuading across Diverse Domains: a Dataset and Persuasion Large Language Model (2024.acl-long)

Challenge:	Existing evaluations of LLMs in finance are text-only, monolingual, and largely saturated by current models.
Approach:	They propose a multilingual and multimodal benchmark for evaluating LLMs in real financial contexts.
Outcome:	The first expert-annotated multilingual and multimodal benchmark is released . it evaluates 21 leading LLMs and shows they perform better in multilingual settings .

Copied to clipboard

Chuhao Jin, Kening Ren, Lingzhen Kong, Xiting Wang, Ruihua Song, Huan Chen

Glue pizza and eat rocks - Exploiting Vulnerabilities in Retrieval-Augmented Generative Models (2024.emnlp-main)

Challenge:	Persuasive dialogue requires multi-turn following and planning abilities to achieve the goal of persuating users.
Approach:	They propose a general method to learn a persuasive model based on LLMs through intent-to-strategy reasoning, which summarizes the intent of user’s utterance and reasons next strategy to respond.
Outcome:	The proposed method outperforms baselines on automatic evaluation metric Win-Rate and human evaluation on two datasets.

Copied to clipboard

Zhen Tan, Chengshuai Zhao, Raha Moraffah, Yifan Li, Song Wang, Jundong Li, Tianlong Chen, Huan Liu

LAM SIMULATOR: Advancing Data Generation for Large Action Model Training via Online Exploration and Trajectory Feedback (2025.findings-acl)

Challenge:	Retrieval-Augmented Generative (RAG) models enhance Large Language Models (LLMs) by integrating external knowledge bases.
Approach:	They propose to exploit openness of RAG models by injecting deceptive content into the retrieval database, intentionally changing the model’s behavior.
Outcome:	The proposed model can be exploited through crafted content uploads with access to the retriever.

Copied to clipboard

Thai Quoc Hoang, Kung-Hsiang Huang, Shirley Kokane, Jianguo Zhang, Zuxin Liu, Ming Zhu, Jake Grigsby, Tian Lan, Michael S Ryoo, Chien-Sheng Wu, Shelby Heinecke, Huan Wang, Silvio Savarese, Caiming Xiong, Juan Carlos Niebles

Challenge:	Large Action Models (LAMs) face challenges due to the need for high-quality training data, especially for multi-steps tasks that involve planning, executing tool calls, and responding to feedback.
Approach:	They propose a framework for online exploration of agentic tasks with high-quality feedback . they use a dynamic task query generator and an extensive collection of tools to create a high-level feedback environment for LLM Agents.
Outcome:	The proposed framework achieves 49.3% performance improvement over baselines on toolbench and CRMArena.

World Models with Hints of Large Language Models for Goal Achieving (2025.naacl-long)

Copied to clipboard

Zeyuan Liu, Ziyu Huan, Xiyao Wang, Jiafei Lyu, Jian Tao, Xiu Li, Furong Huang, Huazhe Xu

LATTE: Learning to Think with Vision Specialists (2025.emnlp-main)

Challenge:	Existing methods address this by adding intrinsic rewards, but they fail to provide meaningful guidance in long-horizon decision-making tasks with large state and action spaces lacking purposeful exploration.
Approach:	They propose a multi-modal model-based RL approach that integrates the proposed hinting subgoals into the model rollouts to encourage goal discovery and reaching in challenging tasks.
Outcome:	The proposed model outperforms existing methods in challenging, sparse-reward environments such as HomeGrid, Crafter, and Minecraft by 41.8%, 21.1%, and 9.9%.

Copied to clipboard

Zixian Ma, Jianguo Zhang, Zhiwei Liu, Jieyu Zhang, Juntao Tan, Manli Shu, Juan Carlos Niebles, Shelby Heinecke, Huan Wang, Caiming Xiong, Ranjay Krishna, Silvio Savarese

SimRPD: Optimizing Recruitment Proactive Dialogue Agents through Simulator-Based Data Evaluation and Selection (2026.acl-industry)

Challenge:	Open-source vision-language models excel on simple question-answering tasks, but struggle with complex questions that require both perception and reasoning.
Approach:	They propose a family of vision-language models that have LeArned to Think wiTh vision spEcialists by offloading perception to state-of-the-art vision models.
Outcome:	The proposed model achieves 4-5% gains over baselines across 6 benchmarks covering both perception and reasoning abilities.

Copied to clipboard

Zhiyong Cao, Dunqiang Liu, Qi Dai, Haojun Xu, Huai Yuen Khor, Hao Wang, Huan He, Yafei Liu, Ke Ma, Ruqian Shi, Sicheng Zhou, Sijia Yao

MCPEval: Automatic MCP-based Deep Evaluation for AI Agent Models (2025.emnlp-demos)

Challenge:	High-quality data in training proactive dialogue agents is scarce, despite fine-tuning and reinforcement learning . a recent study has shown that the effectiveness of supervised fine-touring is limited by the lack of high-quality, domain-specific training data.
Approach:	They propose a framework for training recruitment proactive dialogue agents using a high-fidelity user simulator and a multi-dimensional evaluation framework based on Chain-of-Intention.
Outcome:	The proposed framework outperforms existing simulator-based data selection strategies in a real-world recruitment scenario.

Copied to clipboard

Zhiwei Liu, Jielin Qiu, Shiyu Wang, Jianguo Zhang, Zuxin Liu, Roshan Ram, Haolin Chen, Weiran Yao, Shelby Heinecke, Silvio Savarese, Huan Wang, Caiming Xiong

Challenge:	Existing evaluation frameworks suffer from limitations such as static task benchmarks, limited scope, and inadequate integration with practical applications.
Approach:	They propose an open-source, Model Context Protocol-based evaluation framework specifically tailored for comprehensive and systematic assessment of LLM-powered agents.
Outcome:	The proposed framework uncovers nuanced performance patterns and identify domain-specific strengths and weaknesses, providing valuable insights beyond traditional binary success metrics.

Towards Non-task-specific Distillation of BERT via Sentence Representation Approximation (2020.aacl-main)

Copied to clipboard

Bowen Wu, Huan Zhang, MengYuan Li, Zongsheng Wang, Qihang Feng, Junhong Huang, Baoxun Wang

Automatic Evaluation of Attribution by Large Language Models (2023.findings-emnlp)

Challenge:	Existing methods for transferring knowledge from BERT into a model with large parameters are not efficient due to their large-scale and high computational cost.
Approach:	They propose a sentence representation approximating oriented distillation framework that can distill pre-trained BERT into a simple LSTM based model without specifying tasks.
Outcome:	The proposed model outperforms other distillation methods and larger models on multiple NLP tasks with efficiency well-improved.

Copied to clipboard

Xiang Yue, Boshi Wang, Ziru Chen, Kai Zhang, Yu Su, Huan Sun

Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debate (2023.findings-emnlp)

Challenge:	Generative large language models (LLMs) incorporate external references to generate and support claims. however, evaluating the attribution remains an open problem.
Approach:	They investigate automatic evaluation of attribution given by large language models . they define different types of attributed errors and then explore two approaches .
Outcome:	The proposed methods highlight promising signals and challenges.

Copied to clipboard

Boshi Wang, Xiang Yue, Huan Sun

How do Visual Attributes Influence Web Agents? A Comprehensive Evaluation of User Interface Design Factors (2026.findings-acl)

Challenge:	Large language models (LLMs) have shown impressive performance in complex reasoning tasks, but it is difficult to know whether they are reasoning based on deep understandings of truth and logic or leveraging their vast previously-seen patterns in a relatively shallow way.
Approach:	They propose to test large language models by engaging with them in a debate-like conversation where the user and LLM need to discuss to make the correct decision starting from opposing arguments.
Outcome:	The proposed model can achieve the correct answer on its own, but can also hold and defend its belief instead of blindly believing or getting misled by the user’s (invalid) arguments and critiques.

Copied to clipboard

Kuai Yu, Naicheng Yu, Han Wang, Rui Yang, Huan Zhang

EngiBench: A Benchmark for Evaluating Large Language Models on Engineering Problem Solving (2026.findings-acl)

Challenge:	Existing studies on the effect of environmental variation on web agents have focused on robustness to adversarial attacks with less attention to agents’ preferences in benign scenarios.
Approach:	They propose a controlled evaluation pipeline to quantify how visual attributes influence web-agent decision-making by comparing variants and browsing interactions.
Outcome:	Extensive experiments on 8 variant families, 5 real-world websites and 4 representative web agents show that background color contrast, item size, position, and card clarity have a strong influence on agents’ actions, whereas font styling, text color, and item image clarity exhibit minor effects.

Copied to clipboard

Xiyuan Zhou, Xinlei Wang, Yirui He, Ruixi Zou, Yang Wu, Yuheng Cheng, Yulu Xie, Wenxuan Liu, Huan Zhao, Yan Xu, Jinjin Gu, Junhua Zhao

Reasoning

Challenge:	Existing benchmarks focus on well-defined or abstract reasoning and fail to capture real-world engineering problems.
Approach:	They propose a hierarchical benchmark to evaluate large language models on engineering problems.
Outcome:	The proposed model performs well under well-defined conditions and is based on three levels of difficulty and covers diverse engineering subfields.

Rethinking Stateful Tool Use in Multi-Turn Dialogues: Benchmarks and Challenges (2025.findings-acl)

Copied to clipboard

Hongru Wang, Wenyu Huang, Yufei Wang, Yuanhao Xi, Jianqiao Lu, Huan Zhang, Nan Hu, Zeming Liu, Jeff Z. Pan, Kam-Fai Wong

NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation (2023.acl-long)

Challenge:	Existing benchmarks that assess Language Models (LMs) as Language Agents (LAs) for tool use focus on stateless, single-turn interactions or partial evaluations, overlooking the inherent stateful nature of interactions in multi-turn applications.
Approach:	They propose a multi-turn dialogue dataset with stateful tool interactions considering the whole life cycle of tool use across six key tasks in three stages . they also build VirtualMobile – an embodied virtual mobile evaluation environment to simulate API calls and assess the robustness of the created APIs.
Outcome:	The proposed dataset evaluates 13 open- and closed-source LLMs and provides detailed analysis at each stage.

Copied to clipboard

Shengming Yin, Chenfei Wu, Huan Yang, Jianfeng Wang, Xiaodong Wang, Minheng Ni, Zhengyuan Yang, Linjie Li, Shuguang Liu, Fan Yang, Jianlong Fu, Ming Gong, Lijuan Wang, Zicheng Liu, Houqiang Li, Nan Duan

CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environments (2025.naacl-long)

Challenge:	Existing work generates long videos segment by segment sequentially, which is inefficient.
Approach:	They propose a Diffusion over Difference architecture for eXtremely Long video generation.
Outcome:	The proposed architecture reduces the average inference time from 7.55min to 26s (94.26%) and generates high-quality long videos with both global and local coherence.

Copied to clipboard

Kung-Hsiang Huang, Akshara Prabhakar, Sidharth Dhawan, Yixin Mao, Huan Wang, Silvio Savarese, Caiming Xiong, Philippe Laban, Chien-Sheng Wu

Challenge:	Existing benchmarks for evaluating CRM agents on work-related tasks are limited due to data privacy concerns.
Approach:	They propose a benchmark to evaluate AI agents on real-world CRM tasks . they use 16 commonly used industrial objects with high interconnectivity to simulate real data distributions.
Outcome:	The new benchmark evaluates AI agents on real-world customer service tasks . it includes 16 commonly used industrial objects with high interconnectivity . the results highlight the need for enhanced agent capabilities in function-calling and rule-following .

xLAM: A Family of Large Action Models to Empower AI Agent Systems (2025.naacl-long)

Copied to clipboard

Jianguo Zhang, Tian Lan, Ming Zhu, Zuxin Liu, Thai Quoc Hoang, Shirley Kokane, Weiran Yao, Juntao Tan, Akshara Prabhakar, Haolin Chen, Zhiwei Liu, Yihao Feng, Tulika Manoj Awalgaonkar, Rithesh R N, Zeyuan Chen, Ran Xu, Juan Carlos Niebles, Shelby Heinecke, Huan Wang, Silvio Savarese, Caiming Xiong

Large Language Models for Data Annotation and Synthesis: A Survey (2024.emnlp-main)

Challenge:	Autonomous agents powered by large language models (LLMs) have attracted significant research interest, but there are few standards for developing specialized models for agent tasks.
Approach:	They propose a series of large action models with dense and mixture-of-expert architectures that unifies, augments, and synthesizes diverse datasets to enhance agent generalizability and performance.
Outcome:	The proposed models outperform GPT-4, Claude-3, and many other models in terms of tool use and outperformed GPT-based models on multiple agent ability benchmarks.

Copied to clipboard

Zhen Tan, Dawei Li, Song Wang, Alimohammad Beigi, Bohan Jiang, Amrita Bhattacharjee, Mansooreh Karami, Jundong Li, Lu Cheng, Huan Liu

Improving Event Representation via Simultaneous Weakly Supervised Contrastive Learning and Clustering (2022.acl-long)

Challenge:	Existing surveys focus on LLMs' specific utility for data annotation and synthesis.
Approach:	They propose to use large language models to generate annotations from raw data . they also propose to review learning strategies for models utilizing LLM-generated annotations .
Outcome:	The proposed models can be used to improve the efficacy of machine learning models by generating and labeling raw data with relevant information.

Copied to clipboard

Jun Gao, Wei Wang, Changlong Yu, Huan Zhao, Wilfred Ng, Ruifeng Xu

SlackAgents: Scalable Collaboration of AI Agents in Workspaces (2025.emnlp-demos)

Challenge:	Existing work exploits easily accessible co-occurrence information of events to learn event representations.
Approach:	They propose a weakly supervised contrastive learning method and a prototype-based clustering method for event representation learning.
Outcome:	The proposed framework outperforms baselines on Hard Similarity and Transitive Sentence Similarity tasks.

Copied to clipboard

Zhiwei Liu, Weiran Yao, Zuxin Liu, Juntao Tan, Jianguo Zhang, Frank Wang, Sukhandeep Nahal, Huan Wang, Shelby Heinecke, Silvio Savarese, Caiming Xiong

Robustness in NLP

Challenge:	Existing open-source frameworks like LangChain and LlamaIndex fail to integrate into daily workflows, resulting in limited daily usage for work.
Approach:	They propose a multi-agent library for scalable management and collaboration of AI agents on Slack.
Outcome:	The proposed framework offers instant AI integration into organizational workflows and facilitates scalable collaboration, allowing for effective communication and task orchestration.

See the Forest for the Trees: Loosely Speculative Decoding via Visual-Semantic Guidance for Efficient Inference of Video LLMs (2026.acl-long)

Copied to clipboard

Yicheng Ji, Jun Zhang, Jinpeng Chen, Cong Wang, Lidan Shou, Gang Chen, Huan Li

Context-Driven Index Trimming: A Data Quality Perspective to Enhancing Precision of RALMs (2024.findings-emnlp)

Challenge:	Existing methods for video understanding suffer from autoregressive generation of tokens.
Approach:	They propose a training-free loosely SD framework for Video-LLMs that uses visual-relevant tokens to accurately pinpoint the latter.
Outcome:	The proposed framework boosts the accepted length and speedup ratio by 136% and 35% compared to SOTA training-free SD methods for Video-LLMs.

Copied to clipboard

Kexin Ma, Ruochun Jin, Wang Haotian, Wang Xi, Huan Chen, Yuhua Tang, Qian Wang

Combating Security and Privacy Issues in the Era of Large Language Models (2024.naacl-tutorials)

Challenge:	Existing research often overlooks the data quality issues within retrieval results, often caused by inaccurate existing vector-distance-based retrieval methods.
Approach:	They propose to use Context-Driven Index Trimming framework to capture and regulate consistency between retrieved contexts and modify indexes in the database.
Outcome:	Experiments show that the proposed framework can improve answer quality by 3.75% on open-domain question-answering tasks.

Copied to clipboard

Muhao Chen, Chaowei Xiao, Huan Sun, Lei Li, Leon Derczynski, Anima Anandkumar, Fei Wang

Fuzzy Reasoning Chain (FRC): An Innovative Reasoning Framework from Fuzziness to Clarity (2025.findings-emnlp)

Challenge:	a tutorial aims to provide a summary of risks and vulnerabilities in large language models . a number of studies have focused on security, privacy and copyright aspects of LLMs .
Approach:	This tutorial seeks to provide a systematic summary of risks and vulnerabilities in large language models . authors will discuss security, privacy and copyright aspects of LLMs .
Outcome:	This tutorial aims to provide a systematic summary of risks and vulnerabilities in large language models . it will also outline emerging challenges in security, privacy and reliability of LLMs .

Copied to clipboard

Ping Chen, Xiang Liu, Zhaoxiang Liu, Zezhou Chen, Xingpeng Zhang, Huan Hu, Zipeng Wang, Kai Wang, Shuming Shi, Shiguo Lian