Papers by Pei Wang

Learning Task Decomposition to Assist Humans in Competitive Programming (2024.acl-long)

Challenge:	Existing methods for finding out-of-domain intents suffer from in-domain overfitting problem . previous methods fail to transfer prior knowledge to downstream clustering .
Approach:	They propose a unified K-nearest neighbor contrastive learning framework to discover OOD intents . they propose IND pre-training objective to learn discriminative features while maintaining intra-class diversity .
Outcome:	The proposed framework improves on three benchmark datasets.

Copied to clipboard

Jiaxin Wen, Ruiqi Zhong, Pei Ke, Zhihong Shao, Hongning Wang, Minlie Huang

Fine-tuning Language Models for Joint Rewriting and Completion of Code with Potential Bugs (2024.findings-acl)

Challenge:	Using language models (LMs) to solve complex problems, humans might struggle to understand and repair flawed ones.
Approach:	They propose to automatically decompose complex problems into simpler pieces that correspond to specific subtasks and measure their assistive value.
Outcome:	The proposed method enables non-experts to solve 33.3% more problems and speeds them up by 3.3x .

Copied to clipboard

Dingmin Wang, Jinman Zhao, Hengzhi Pei, Samson Tan, Sheng Zha

Text Generation

Challenge:	Previous work has demonstrated shortcomings of large language models of code (CodeLLMs) in completing drafty partial code with potential bugs.
Approach:	They propose to use large language models of code to fine-tune their models to rewrite and complete drafty partial code into functional full programs.
Outcome:	The proposed approach achieves superior pass rates over baselines and preserves the integrity of the original partial implementations.

Mitigating Geospatial Knowledge Hallucination in Large Language Models: Benchmarking and Dynamic Factuality Aligning (2025.findings-emnlp)

Copied to clipboard

Shengyuan Wang, Jie Feng, Tianhui Liu, Dan Pei, Yong Li

CAT: Confidence-Adaptive Thinking for Efficient Reasoning of Large Reasoning Models (2026.acl-industry)

Challenge:	Large language models (LLMs) have extensive world knowledge, but often generate inaccurate geospatial knowledge.
Approach:	They propose a framework for evaluation of large language models to mitigate hallucinations . they use Kahneman-Tversky Optimization to align LLMs with their reality .
Outcome:	The proposed evaluation framework uncovers hallucinations in 20 advanced LLMs.

Copied to clipboard

Qizhi Jiang, Shuo Wang, Pei Ke, Yuhang Song, Ke Qin

ChartVerse: Scaling Chart Reasoning via Reliable Programmatic Synthesis from Scratch (2026.acl-long)

Challenge:	Existing compression methods for large reasoning models rely on uniform length reduction or coarse-grained difficulty estimation, often leading to performance degradation on difficult problems.
Approach:	They propose a framework that incorporates model’s intrinsic self-certainty signals as confidence into the preference optimization process, which autonomously modulates reasoning lengths based on problem difficulty.
Outcome:	The proposed framework outperforms state-of-the-art models on reasoning accuracy across multiple benchmarks on different base models.

Copied to clipboard

Zheng Liu, Honglin Lin, Xiaoyang Wang, Xin Gao, Yu Li, Mengzhang Cai, Yun Zhu, Zhanping Zhong, Qizhi Pei, Zhuoshi Pan, Xiaoran Shang, Conghui He, Bin Cui, Wentao Zhang, Lijun Wu

DocLLM: A Layout-Aware Generative Language Model for Multimodal Document Understanding (2024.acl-long)

Challenge:	Existing open-source vision language models lack high-quality training data for chart reasoning . current models are simplistic and repetitive, while associated QA pairs are prone to hallucinations .
Approach:	They propose a framework to synthesize complex charts and reliable reasoning data from scratch.
Outcome:	Experimental results show that ChartVerse-8B surpasses existing models in QA and difficulty . lack of high-quality training data hampers development of open-source models .

Copied to clipboard

Dongsheng Wang, Natraj Raman, Mathieu Sibue, Zhiqiang Ma, Petr Babkin, Simerjot Kaur, Yulong Pei, Armineh Nourbakhsh, Xiaomo Liu

Aligning Large Language Models with Implicit Preferences from User-Generated Content (2025.acl-long)

Challenge:	Documents with rich layouts are a significant portion of enterprise corpora and document AI is still a challenge.
Approach:	They propose a lightweight extension to traditional large language models for reasoning over visual documents that takes into account both textual semantics and spatial layout.
Outcome:	The proposed model outperforms existing large language models on 14 out of 16 datasets and generalizes well to 4 out of 5 previously unseen datasets.

Copied to clipboard

Zhaoxuan Tan, Zheng Li, Tianyi Liu, Haodong Wang, Hyokun Yun, Ming Zeng, Pei Chen, Zhihan Zhang, Yifan Gao, Ruijie Wang, Priyanka Nigam, Bing Yin, Meng Jiang

Multimodal Large Language Models for Text-rich Image Understanding: A Comprehensive Review (2025.findings-acl)

Challenge:	Existing preference learning methods rely heavily on curated data from humans or advanced LLMs, which is costly and difficult to scale.
Approach:	They propose a framework that leverages implicit preferences in unlabeled user-generated content to generate preference data.
Outcome:	The proposed framework transforms user-generated content into user queries and generates responses from the policy model.

Copied to clipboard

Pei Fu, Tongkun Guan, Zining Wang, Zhentao Guo, Chen Duan, Hao Sun, Boming Chen, Qianyi Jiang, Jiayao Ma, Kai Zhou, Junfeng Luo

Generalized Intent Discovery: Learning from Open World Dialogue System (2022.coling-1)

Challenge:	Recent advances in vision-language models have unified perception and understanding tasks within Visual Question Answering paradigms.
Approach:	They propose to outline timeline, architecture, and pipeline of nearly all TIU MLLMs and review their performance on mainstream benchmarks.
Outcome:	The proposed models perform well on mainstream benchmarks and are compared with other models.

Copied to clipboard

Yutao Mou, Keqing He, Yanan Wu, Pei Wang, Jingang Wang, Wei Wu, Yi Huang, Junlan Feng, Weiran Xu

One Battle After Another: Probing LLMs’ Limits on Multi-Turn Instruction Following with a Benchmark Evolving Framework (2026.acl-long)

Challenge:	Existing intent classification models rely on a pre-defined intent set and supervised labels, which is limited in some practical scenarios.
Approach:	They propose to extend an IND intent classifier to an open-world intent set including IND and OOD intents.
Outcome:	The proposed task can classify IND and OOD intents while discovering new unlabeled OOD types incrementally.

Copied to clipboard

Qi Jia, Ye Shen, Xiujie Song, Kaiwei Zhang, Shibo Wang, Dun Pei, Xiangyang Zhu, Guangtao Zhai

ALERT: An LLM-powered Benchmark for Automatic Evaluation of Recommendation Explanations (2025.naacl-long)

Challenge:	Existing benchmarks for instruction-following in multi-topic dialogues are limited to a fixed number of turns, susceptible to saturation and failing to account for users’ interactive experience.
Approach:	They propose a framework featuring a three-layer tracking mechanism and a query synthesis agent to mimic sequential user behaviors.
Outcome:	The proposed framework outperforms existing benchmarks in the evaluation of instruction following in multi-topic dialogues and demonstrates deficiencies in failure recovery and fine-grained instruction following.

Copied to clipboard

Yichuan Li, Xinyang Zhang, Chenwei Zhang, Mao Li, Tianyi Liu, Pei Chen, Yifan Gao, Kyumin Lee, Kaize Ding, Zhengyang Wang, Zhihan Zhang, Jingbo Shang, Xian Li, Trishul Chilimbi

EVM-QuestBench: An Execution-Grounded Benchmark for Natural-Language Transaction Code Generation (2026.acl-long)

Challenge:	Existing benchmarks for recommendation explanation evaluation lack item diversity and user preferences data.
Approach:	They propose a model-agnostic recommendation explanation evaluation benchmark based on Amazon e-commerce categories with implicit preferences . they propose two novel automatic evaluators that enable scalable and human-preference aligned evaluation of explanations .
Outcome:	The proposed model-agnostic evaluation benchmark outperforms existing methods in a variety of domains.

Copied to clipboard

Pei Yang, Wanyi Chen, Ke Wang, Lynn Ai, Eric Yang, Tianyu Shi

Programming Languages in NLP

Challenge:	Existing evaluations of large language models overlook execution accuracy and safety.
Approach:	They propose an execution-grounded benchmark for natural-language transaction-script generation on EVM-compatible chains.
Outcome:	The proposed benchmark finds large performance gaps in the models with 5 independent rounds.

ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectors (2024.findings-emnlp)

Copied to clipboard

Zhexin Zhang, Yida Lu, Jingyuan Ma, Di Zhang, Rui Li, Pei Ke, Hao Sun, Lei Sha, Zhifang Sui, Hongning Wang, Minlie Huang

Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training (2025.naacl-long)

Challenge:	Existing tools for detecting safety issues in LLMs are expensive and inefficient.
Approach:	They propose an LLM-based safety detector which annotates the safety of queries and provides explanations for its decisions.
Outcome:	The proposed detector outperforms baselines on four sets of query-response pairs and is effective as a safety evaluator for advanced LLMs.

Copied to clipboard

Yuchen Zhuang, Jingfeng Yang, Haoming Jiang, Xin Liu, Kewei Cheng, Sanket Lokegaonkar, Yifan Gao, Qing Ping, Tianyi Liu, Binxuan Huang, Zheng Li, Zhengyang Wang, Pei Chen, Ruijie Wang, Rongzhi Zhang, Nasser Zalmout, Priyanka Nigam, Bing Yin, Chao Zhang

MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing (2026.acl-industry)

Challenge:	Existing LLMs often rely on complex prompting or extensive fine-tuning to introduce new capabilities while preserving strong generalizability.
Approach:	They propose a large-scale pre-training corpus to enhance LLM agents' capabilities . they use 103B agent-specific data encompassing 76,537 APIs .
Outcome:	The proposed training corpus outperforms open-source LLMs and commercial LLM agents on three agent benchmarks.

Copied to clipboard

Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wenzheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Zirui Tang, Boyu Niu, Yuanhong Zheng, Dongsheng Ma, Ziyang Miao, Hejun Dong, Siyi Qian, Junyuan Zhang, Fangdong Wang, Jingzhou Chen, Xiaomeng Zhao, Liqun Wei, Wei Li, Shasha Wang, RuiLiang Xu, Yuanyuan Cao, Lu Chen, Qianqian Wu, Huaiyu Gu, Lindong Lu, Dechen Lin, null Shenguanlin, Xuanhe Zhou, Linfeng Zhang, Yuhang Zang, Xiaoyi Dong, Jiaqi Wang, Bo Zhang, Lei Bai, Pei Chu, Weijia Li, Jiang Wu, Lijun Wu, Zhenxiang Li, Guangyu Wang, Zhongying Tu, Chao Xu, Kai Chen, Bowen Zhou, Dahua Lin, Wentao Zhang, Conghui He

Green & Sustainable NLP

Challenge:	Document images are characterized by higher resolutions, denser content, and more complex structural layouts.
Approach:	They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition.
Outcome:	The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead.

Meta-Reasoning: Semantics-Symbol Deconstruction for Large Language Models (2024.findings-acl)

Copied to clipboard

Yiming Wang, Zhuosheng Zhang, Pei Zhang, Baosong Yang, Rui Wang

AutoDetect: Towards a Unified Framework for Automated Weakness Detection in Large Language Models (2024.findings-emnlp)

Challenge:	Existing methods rely on syntactically mapping natural languages to complete formal languages like Python and SQL.
Approach:	They propose to deconstruct reasoning-independent semantic information into generic symbolic representations, thereby efficiently capturing more generalized reasoning knowledge.
Outcome:	The proposed method improves in-context reasoning accuracy, learning efficiency, out-of-domain generalization, and output stability compared to the Chain-of thought technique.

Copied to clipboard

Jiale Cheng, Yida Lu, Xiaotao Gu, Pei Ke, Xiao Liu, Yuxiao Dong, Hongning Wang, Jie Tang, Minlie Huang

UniConv: Unifying Retrieval and Response Generation for Large Language Models in Conversations (2025.acl-long)

Challenge:	Large Language Models (LLMs) exhibit significant but subtle weaknesses, such as mistakes in instruction-following or coding tasks.
Approach:	They propose a framework to automatically expose weaknesses in Large Language Models (LLMs) they use three LLM-powered agents to perform comprehensive weakness identification .
Outcome:	The proposed framework shows that it is more effective than untargeted data augmentation methods like Self-Instruct to identify weaknesses in LLMs.

Copied to clipboard

Fengran Mo, Yifan Gao, Chuan Meng, Xin Liu, Zhuofeng Wu, Kelong Mao, Zhengyang Wang, Pei Chen, Zheng Li, Xian Li, Bing Yin, Meng Jiang

UniNL: Aligning Representation Learning with Scoring Function for OOD Detection via Unified Neighborhood Learning (2022.emnlp-main)

Challenge:	Existing conversational search systems are usually built with two different models . this separation restricts the system from leveraging the model's intrinsic knowledge simultaneously . Existing studies for developing unified models cannot fully address the aspects of understanding conversational context, managing retrieval independently, and generating responses.
Approach:	They propose to unify dense retrieval and response generation for large language models in conversation by fine-tuning and mitigating data discrepancy.
Outcome:	The proposed model can outperform existing models on five conversational search datasets and reduce inconsistency risks while mitigating data discrepancy.

Copied to clipboard

Yutao Mou, Pei Wang, Keqing He, Yanan Wu, Jingang Wang, Wei Wu, Weiran Xu

ShopSimulator: Evaluating and Exploring RL-Driven LLM Agent for Shopping Assistants (2026.acl-long)

Challenge:	Existing methods to detect out-of-domain (OOD) intents ignore alignment between representation learning and scoring function, limiting performance.
Approach:	They propose a unified neighborhood learning framework to detect OOD intents . they propose to align representation learning with scoring function .
Outcome:	The proposed method is able to detect out-of-domain (OOD) intents from user queries.

Copied to clipboard

Pei Wang, Yanan Wu, Xiaoshuai Song, Weixun Wang, Gengru Chen, Zhongwen Li, Kezhong Yan, Qi Liu, Ken Deng, Shuaibing Zhao, Shaopan Xiong, Xuepeng Liu, Xuefeng Chen, Wanxi Deng, Wenbo Su, Bo Zheng

Challenge:	Existing studies on large language model-based agents focus on evaluation benchmarks without training support.
Approach:	They propose a large-scale Chinese shopping simulation environment that uses large language models to train agents.
Outcome:	The proposed model performs poorly in a large-scale and challenging shopping environment in China.

AlignBench: Benchmarking Chinese Alignment of Large Language Models (2024.acl-long)

Copied to clipboard

Xiao Liu, Xuanyu Lei, Shengyuan Wang, Yue Huang, Andrew Feng, Bosi Wen, Jiale Cheng, Pei Ke, Yifan Xu, Weng Lam Tam, Xiaohan Zhang, Lichao Sun, Xiaotao Gu, Hongning Wang, Jing Zhang, Minlie Huang, Yuxiao Dong, Jie Tang

DecompEval: Evaluating Generated Texts as Unsupervised Decomposed Question Answering (2023.acl-long)

Challenge:	Effective evaluation of alignment for emerging Chinese LLMs is still significantly lacking, calling for real-scenario grounded, open-ended, challenging and automatic evaluations tailored for alignment.
Approach:	They propose a multi-dimensional benchmark for evaluating LLMs’ alignment in Chinese with 8 main categories, 683 real-scenario rooted queries and corresponding human verified references.
Outcome:	The benchmark uses a human-in-the-loop data curation pipeline, 683 real-scenario rooted queries and human verified references.

Copied to clipboard

Pei Ke, Fei Huang, Fei Mi, Yasheng Wang, Qun Liu, Xiaoyan Zhu, Minlie Huang

STORM-BORN: A Challenging Mathematical Derivations Dataset Curated via a Human-in-the-Loop Multi-Agent Framework (2025.findings-acl)

Challenge:	Existing evaluation metrics for natural language generation (NLG) tasks face the challenges on generalization ability and interpretability.
Approach:	They propose a metric that evaluates natural language generation tasks as an instruction-style question answering task and utilizes instruction-tuned pre-trained language models without training on evaluation datasets.
Outcome:	The proposed metric achieves state-of-the-art performance in untrained metrics for evaluating text summarization and dialogue generation, which exhibits strong dimension-level / task-level generalization ability and interpretability.

Copied to clipboard

Wenhao Liu, Zhenyi Lu, Xinyu Hu, Jerry Zhang, Dailin Li, Jiacheng Cen, Huilin Cao, Haiteng Wang, Yuhan Li, Xie Kun, Dandan Li, Pei Zhang, Chengbo Zhang, Yuxiang Ren, Xiaohong Huang, Yan Ma

LongLeader: A Comprehensive Leaderboard for Large Language Models in Long-context Scenarios (2025.naacl-long)

Challenge:	Existing datasets suffer from outdated and insufficient challenging content, neglecting human-like reasoning, and limited reliability due to single-LLM generation.
Approach:	They propose a human-in-the-loop, multi-agent data generation framework that integrates reasoning-dense filters, multiagent collaboration, and human mathematicians’ evaluations to ensure the reliability and quality of the dataset.
Outcome:	The proposed framework improves accuracy and quality of the 2,000-synthesized datasets by integrating reasoning-dense filters, multi-agent collaboration, and human mathematicians’ evaluations.

Copied to clipboard

Pei Chen, Hongye Jin, Cheng-Che Lee, Rulin Shao, Jingfeng Yang, Mingyu Zhao, Zhaoyu Zhang, Qin Lu, Kaiwen Men, Ning Xie, Huasheng Li, Bing Yin, Han Li, Lingyun Wang

Large Language Model for Multi-Domain Translation: Benchmarking and Domain CoT Fine-tuning (2024.findings-emnlp)

Challenge:	LongLeader aims to assess different LLMs' long-context comprehension abilities . long-constext comprehension is a key bottleneck for many use cases .
Approach:	They propose a leaderboard to assess different LLMs' long-context comprehension abilities . they offer open-source access to the benchmarks and maintain a dedicated website .
Outcome:	The proposed model assesses different LLMs on selected benchmarks and provides open-source access to the benchmarks.

Copied to clipboard

Tianxiang Hu, Pei Zhang, Baosong Yang, Jun Xie, Derek Wong, Rui Wang

Trajectory2Task: Training Robust Tool-Calling Agents with Synthesized Yet Verifiable Data for Complex User Intents (2026.acl-long)

Challenge:	Achieving consistent high-quality machine translation across diverse domains remains a challenge due to limited and imbalanced parallel training data available in various domains.
Approach:	They propose a domain Chain of Thought technique that uses the multi-domain intelligence of LLMs to improve translation performance.
Outcome:	The proposed method achieves significant improvements in translation accuracy and domain robustness over traditional fine-tuning on a small dataset of four domains.

Copied to clipboard

Ziyi Wang, Yuxuan Lu, Yimeng Zhang, Pei Chen, Ziwei Dong, Jing Huang, Jiri Gesi, Xianfeng Tang, Chen Luo, Qun Liu, Yisi Sang, Hanqing Lu, Manling Li, Jin Lai, Dakuo Wang

Improving Large Language Models Function Calling and Interpretability via Guided-Structured Templates (2025.emnlp-main)

Challenge:	Tool-calling agents are increasingly deployed in real-world customer-facing workflows . but most studies on tool-callers focus on idealized settings with general, fixed, and well-specified tasks.
Approach:	They propose a tool-calling agent-based data pipeline that converts trajectories into user-facing tasks with controlled intent adaptations.
Outcome:	The proposed pipeline can be used to study tool use under three scenarios.

Copied to clipboard

Hy Dang, Tianyi Liu, Zhuofeng Wu, Jingfeng Yang, Haoming Jiang, Tao Yang, Pei Chen, Zhengyang Wang, Helen Wang, Huasheng Li, Bing Yin, Meng Jiang

A Systematic Assessment of Language Models with Linguistic Minimal Pairs in Chinese (2026.tacl-1)

Challenge:	Large language models (LLMs) have strong reasoning and tool-use capabilities, yet fail in real-world tool-interactions due to incorrect parameterization, poor tool selection, or misinterpretation of user intent.
Approach:	They propose a curriculum-inspired framework that leverages structured reasoning templates to guide LLMs through more deliberate step-by-step instructions for generating function calls.
Outcome:	The proposed framework reduces tool-use errors and improves interpretability and transparency of tool-using agents.

Copied to clipboard

Yikang Liu, Yeting Shen, Hongao Zhu, Lilong Xu, Zhiheng Qian, Siyuan Song, Kejia Zhang, Jialong Tang, Pei Zhang, Baosong Yang, Rui Wang, Hai Hu

Training Language Model to Critique for Better Refinement (2025.findings-acl)

Challenge:	Using sub-linear length normalized log-probabilities (SLLN-LP), we find unequal lengths of sentences in minimal pairs difficult for LMs even up to 32B parameters.
Approach:	They propose to use ZhoBLiMP as a linguistic minimal pair benchmark for Chinese language models to mitigate biases.
Outcome:	The proposed metric mitigates biases in Chinese language models with over 100 paradigms . Anaphor, Quantifiers, and Ellipsis are difficult for LMs even up to 32B parameters .

Copied to clipboard

Tianshu Yu, Chao Xiang, Mingchuan Yang, Pei Ke, Bosi Wen, Cunxiang Wang, Jiale Cheng, Li Zhang, Xinyu Mu, Chuxiong Sun, Minlie Huang

Intent Contrastive Learning Based on Multi-view Augmentation for Sequential Recommendation (2025.coling-main)

Challenge:	Large language models (LLMs) have remarkable evaluation and critique capabilities, providing insightful feedback and identifying flaws in various tasks.
Approach:	They propose a framework to train critic models using refinement signals to generate feedback loops where critiques guide the model in refining its responses.
Outcome:	The proposed framework outperforms traditional methods and open-source models in terms of critique quality and refinement outcomes.

Copied to clipboard

Bo Pei, Yingzheng Zhu, Guangjin Wang, Huajuan Duan, Wenya Wu, Fuyong Xu, Yizhao Zhu, Peiyu Liu, Ran Lu

DolphCoder: Echo-Locating Code Large Language Models with Diverse and Multi-Objective Instruction Tuning (2024.acl-long)

Challenge:	Existing work on intent-related models fails to capture long-term dependencies in user behavior and fails to effectively utilize item relevance.
Approach:	They propose a sequential recommendation framework that combine temporal variability with position encoding that has extrapolation properties to encode sequences, thereby expanding the model’s view of user behavior.
Outcome:	The proposed model improves on three real datasets by 0.8% to 14.7% compared to baselines.

Copied to clipboard

Yejie Wang, Keqing He, Guanting Dong, Pei Wang, Weihao Zeng, Muxi Diao, Weiran Xu, Jingang Wang, Mengdi Zhang, Xunliang Cai

Black-Box Prompt Optimization: Aligning Large Language Models without Model Training (2024.acl-long)

Challenge:	Numerous code large language models (LLMs) have been proposed to enhance code generation performance.
Approach:	They propose a diverse instruction model DolphCoder with self-evaluating for code generation that learns diverse instruction targets and combines a code evaluation objective to enhance its code generation ability.
Outcome:	The proposed model achieves superior performance on the HumanEval and MBPP benchmarks, demonstrating new insights for future code instruction tuning work.

Copied to clipboard

Jiale Cheng, Xiao Liu, Kehan Zheng, Pei Ke, Hongning Wang, Yuxiao Dong, Jie Tang, Minlie Huang

Mosaic-IT: Cost-Free Compositional Data Synthesis for Instruction Tuning (2025.findings-acl)

Challenge:	Large language models are often not well aligned with human intents, which requires additional training.
Approach:	They propose to use Black-Box Prompt Optimization (BPO) to perform alignments on large language models that are not well aligned with human intents.
Outcome:	The proposed model outperforms existing models and is model-agnostic.

Copied to clipboard

Ming Li, Pei Chen, Chenguang Wang, Hongyu Zhao, Yijun Liang, YuPeng Hou, Fuxiao Liu, Tianyi Zhou

R3: End-to-End Reasoning-based Planning for Multi-step Retrosynthesis via Reinforcement Learning (2026.acl-long)

Challenge:	Current instruction tuning relies on teacher models or human intervention to generate and refine the instructions and responses for training, which are costly, non-sustainable, and may lack diversity.
Approach:	They propose a human/model-free compositional data synthesis method that can create rich and diverse augmentations from existing instruction tuning data to enhance large language models.
Outcome:	The proposed method improves performance over benchmarks and reduces training costs by 80% compared with original instruction tuning.

Copied to clipboard

YiFei Wang, Qizhi Pei, Jiangtao Feng, Yuntian Shi, Yi Duan, Lihao Wang, Lei Bai, Lijun Wu, Wei-Ying Ma, Hao Zhou

Large Language Models Meet Open-World Intent Discovery and Recognition: An Evaluation of ChatGPT (2023.emnlp-main)

Challenge:	Experimental results show that R3 is a superior alternative to traditional search algorithms for multistep retrosynthesis planning.
Approach:	They propose a framework that reformulates multistep retrosynthetic planning as a generative reasoning task.
Outcome:	The proposed framework achieves state-of-the-art Top-1 accuracy of 43.7% on retrobench . it leverages Large Language Models to reformulate multistep retrosynthesis as a generative reasoning task.

Copied to clipboard

Xiaoshuai Song, Keqing He, Pei Wang, Guanting Dong, Yutao Mou, Jingang Wang, Yunsen Xian, Xunliang Cai, Weiran Xu

ReCaLL: Membership Inference via Relative Conditional Log-Likelihoods (2024.emnlp-main)

Challenge:	Existing methods to fine-tune discriminative models address these challenges by focusing on in-domain intents.
Approach:	They evaluate ChatGPT on OOD intent discovery and generalized intent discovery tasks . they outline the strengths and weaknesses of ChatGPt and outline their results .
Outcome:	The proposed task aims to extend a closed intent classifier to open-world intent sets.

Copied to clipboard

Roy Xie, Junlin Wang, Ruomin Huang, Minxing Zhang, Rong Ge, Jian Pei, Neil Gong, Bhuwan Dhingra

Continual Generalized Intent Discovery: Marching Towards Dynamic and Open-world Intent Recognition (2023.findings-emnlp)

Challenge:	ReCaLL (Relative Conditional Log-Likelihood) is a membership inference attack that can detect LLMs’ pretraining data by leveraging their conditional language modeling capabilities.
Approach:	They propose a membership inference attack to detect LLMs’ pretraining data by leveraging their conditional language modeling capabilities.
Outcome:	The proposed model achieves state-of-the-art performance on the WikiMIA dataset, even with random and synthetic prefixes, and can be further improved using an ensemble approach.

Copied to clipboard

Xiaoshuai Song, Yutao Mou, Keqing He, Yueyan Qiu, Jinxu Zhao, Pei Wang, Weiran Xu

Reconstructing Event Regions for Event Extraction via Graph Attention Networks (2020.aacl-main)

Challenge:	Currently, the generalized intent classification system only considers one stage of OOD learning and requires all IND data for joint training.
Approach:	They propose a task that detects OOD intents from dynamic OOD data streams . they propose CGID method that bootstraps new intent discovery through class prototypes .
Outcome:	The proposed task can detect out-of-domain (OOD) queries and extend them to the in-domain classifier . it can safely and efficiently detect out of-domain queries and avoid wrong operations .

Copied to clipboard

Pei Chen, Hang Yang, Kang Liu, Ruihong Huang, Yubo Chen, Taifeng Wang, Jun Zhao

Information Extraction & Text Mining

Challenge:	Existing approaches for event extraction focus on sentence-level event extraction, but they lack a broader view of the document context.
Approach:	They build graphs with candidate event filler extractions enriched by sentential embeddings as nodes and use graph attention networks to identify event regions in a document and aggregate event information.
Outcome:	The proposed method performs well on two languages and shows that it is faster than previous methods.

IF-CRITIC: Towards a Fine-Grained LLM Critic for Instruction-Following Evaluation (2026.acl-long)

Copied to clipboard

Bosi Wen, Yilin Niu, Cunxiang Wang, Pei Ke, Xiaoying Ling, Ying Zhang, Aohan Zeng, Hongning Wang, Minlie Huang

Challenge:	Existing evaluation models for instruction-following have many shortcomings, such as substantial costs and unreliable assessments.
Approach:	They propose an LLM critic for fine-grained instruction-following evaluation using a checklist generator and a constraint-level preference optimization method.
Outcome:	The proposed model beats strong LLM-as-a-Judge baselines in evaluations under lower computational overhead compared to baselines.

Beyond the Known: Investigating LLMs Performance on Out-of-Domain Intent Detection (2024.lrec-main)

Copied to clipboard

Pei Wang, Keqing He, Yejie Wang, Xiaoshuai Song, Yutao Mou, Jingang Wang, Yunsen Xian, Xunliang Cai, Weiran Xu

Sentiment Analysis

Challenge:	Out-of-domain (OOD) intent detection is crucial for task-oriented dialogue systems.
Approach:	They conduct a comprehensive evaluation of large language models (LLMs) under various experimental settings and outline their strengths and weaknesses.
Outcome:	The proposed models exhibit strong zero-shot and few-shot capabilities, but is still at a disadvantage compared to models fine-tuned with full resource.

IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluation (2026.acl-long)

Copied to clipboard

Bosi Wen, Yilin Niu, Cunxiang Wang, Xiaoying Ling, Ying Zhang, Pei Ke, Hongning Wang, Minlie Huang

Challenge:	Existing benchmarks for instruction-following lack data coverage and oversimplified pairwise evaluation paradigms that misalign with model optimization scenarios.
Approach:	They propose a meta-evaluation benchmark for instruction-following that covers diverse instruction and constraint types and a preference graph for each instruction.
Outcome:	Extensive experiments on IF-RewardBench show that the proposed benchmark achieves a stronger positive correlation with downstream task performance compared to existing benchmarks.

SAMem: State-Aware Memory as a Fine-Grained Memory for LLM Agents in Decision-Making (2026.findings-acl)

Copied to clipboard

Tong Wang, Pei Xu, Shiyue Cao, Likun Yang, Daipeng Li, Jianbin Jiao, Kaiqi Huang

Probing into the Root: A Dataset for Reason Extraction of Structural Events from Financial Documents (2021.eacl-main)

Challenge:	Existing experiential memory approaches rely on task-level memory, but this lacks the situational alignment required for complex multi-step decision-making.
Approach:	They propose a new fine-grained memory paradigm that aligns memory retrieval with the current state instead of storing and reusing globally shared experiences.
Outcome:	Experiments on complex decision-making benchmarks show that the proposed state-aware memory outperforms existing experiential memory approaches and significantly improves task-solving efficiency.

Copied to clipboard

Pei Chen, Kang Liu, Yubo Chen, Taifeng Wang, Jun Zhao

Information Extraction & Text Mining

Challenge:	Existing methods for event reason extraction are far from resolving this problem.
Approach:	They propose a task to extract causal explanations from document-level texts . they use a dataset FinReason for evaluation to provide Reasons annotation for financial events .
Outcome:	The proposed task performs better than existing methods on a dataset of 8,794 documents, 12,861 financial events and 11,006 reason spans.

LEAF: Large Language Diffusion Model for Time Series Forecasting (2025.findings-emnlp)

Copied to clipboard

Yuhang Pei, Tao Ren, Yifan Wang, Zhipeng Sun, Wei Ju, Chong Chen, Xian-Sheng Hua, Xiao Luo

Long-context Language Models Fail in Basic Retrieval Tasks Without Sufficient Reasoning Steps (2025.findings-emnlp)

Challenge:	Recent work has applied large language models (LLMs) into time series forecasting, but they lack an understanding of holistic temporal patterns with potential error accumulation.
Approach:	They propose a framework that marries Larg e Langu age Diffusion Model with time series forecasting (LEAF) they propose converting time series into tokens and adopting language diffusion models to capture temporal dependencies.
Outcome:	The proposed framework generates future predictions with a diffusion model from a holistic view.

Copied to clipboard

Yijiong Yu, Zhixiao Qi, Yongfeng Huang, Wei Wang, null Weifeng.liu, Ran Chen, Ji Pei

JointGT: Graph-Text Joint Representation Learning for Text Generation from Knowledge Graphs (2021.findings-acl)

Challenge:	despite their extensive context window, long-context language models fail in some basic cases . a recent study shows that long-cot methods are not necessary for long-constituency tasks .
Approach:	a new study evaluates long-context language models with a large context window . the authors propose a method that can be well addressed with arbitrary reasoning steps .
Outcome:	The proposed methods are well addressed with a sufficient number of reasoning steps, guided by specific CoT prompts.

Copied to clipboard

Pei Ke, Haozhe Ji, Yu Ran, Xin Cui, Liwei Wang, Linfeng Song, Xiaoyan Zhu, Minlie Huang

From Experts to Bases: Orthogonal Subspace Mixture for Continual Multimodal Instruction Tuning (2026.acl-long)

Challenge:	Existing pre-trained models for knowledgegraph-to-text generation ignore graph structure during encoding and lack elaborate pre-training tasks to explicitly model graph-text alignments.
Approach:	They propose a graph-text joint representation learning model called JointGT which incorporates a structure-aware semantic aggregation module into each Transformer layer to preserve the graph structure.
Outcome:	The proposed model achieves state-of-the-art performance on various KG-to-text datasets.

Copied to clipboard

Pei Chen, Xilai Wang, null Shiqixu, Zejian Li, Lingyun Sun

Decoupling Pseudo Label Disambiguation and Representation Learning for Generalized Intent Discovery (2023.acl-long)

Challenge:	Existing parameter-efficient approaches to multimodal Continual Instruction Tuning suffer from knowledge interference and inefficient capacity expansion, limiting scalability.
Approach:	They propose a framework for multimodal Continual instruction tuning that decomposes adaptation weights into a globally shared pool of orthonormal bases to capture task-invariant knowledge.
Outcome:	Experiments show that MoBLoRA outperforms state-of-the-art methods while maintaining superior parameter efficiency.

Copied to clipboard

Yutao Mou, Xiaoshuai Song, Keqing He, Chen Zeng, Pei Wang, Jingang Wang, Yunsen Xian, Weiran Xu

POTATO: The Portable Text Annotation Tool (2022.emnlp-demos)

Challenge:	Existing methods for generalized intent discovery lack pseudo label disambiguation and representation learning.
Approach:	They propose a prototype learning framework to decouple pseudo label disambiguation and representation learning.
Outcome:	The proposed method can decouple pseudo label disambiguation and representation learning.

Copied to clipboard

Jiaxin Pei, Aparna Ananthasubramaniam, Xingyao Wang, Naitian Zhou, Apostolos Dedeloudis, Jackson Sargent, David Jurgens

Challenge:	POTATO is a free, fully open-sourced annotation system that supports labeling many types of text and multimodal data.
Approach:	They propose to use POTATO to design and deploy complex annotation tasks.
Outcome:	The proposed annotation system improves labeling speed and productivity over two tasks.

APP: Adaptive Prototypical Pseudo-Labeling for Few-shot OOD Detection (2023.findings-emnlp)

Copied to clipboard

Pei Wang, Keqing He, Yutao Mou, Xiaoshuai Song, Yanan Wu, Jingang Wang, Yunsen Xian, Xunliang Cai, Weiran Xu

Low-Resource NLP

Challenge:	Existing methods for detecting out-of-domain (OOD) intents are hard to label . previous studies use labeled in-domain data to learn intent representations .
Approach:	They propose a prototypical pseudo-labeling method for few-shot OOD detection . they propose 'protoOOD' framework and adaptive pseudo-labeled method .
Outcome:	The proposed method is able to detect out-of-domain (OOD) intents from user queries.

Translationese-index: Using Likelihood Ratios for Graded and Generalizable Measurement of Translationese (2025.emnlp-main)

Copied to clipboard

Yikang Liu, Wanyang Zhang, Yiming Wang, Jialong Tang, Pei Zhang, Baosong Yang, Fei Huang, Rui Wang, Hai Hu

Machine Translation

Challenge:	Translationese is a linguistic property that is often introduced in the translation process that is different from those of original texts.
Approach:	They propose to use synthesized translations and translations in the wild to evaluate T-index's generalizability in cross-domain settings and its validity against human judgments.
Outcome:	The proposed measure can generalize to unseen genres, authors, and language pairs.

DREAM: Disentangling Risks to Enhance Safety Alignment in Multimodal Large Language Models (2025.naacl-long)

Copied to clipboard

Jianyu Liu, Hangyu Guo, Ranjie Duan, Xingyuan Bu, Yancheng He, Shilong Li, Hui Huang, Jiaheng Liu, Yucheng Wang, Chenchen Jing, Xingwei Qu, Xiao Zhang, Pei Wang, Yanan Wu, Jihao Gu, Yangguang Li, Jianke Zhu

Conversational Education at Scale: A Multi-LLM Agent Workflow for Procedural Learning and Pedagogic Quality Assessment (2025.findings-emnlp)

Challenge:	Multimodal Large Language Models (MLLMs) pose unique safety challenges due to their integration of visual and textual data.
Approach:	They propose a method to disentangle risks through step-by-step reasoning within multimodal inputs.
Outcome:	The proposed approach improves safety alignment in MLLMs by fine-tuning and iterative Reinforcement Learning from AI feedback.

Copied to clipboard

Jiahuan Pei, Fanghua Ye, Xin Sun, Wentao Deng, Koen Hindriks, Junxiao Wang

Enhancing Machine Translation with Self-Supervised Preference Data (2025.acl-long)

Challenge:	Existing work on large language models lacks scalability and assesses pedagogic quality.
Approach:	They propose a multi-agent workflow leveraging large language models to simulate interactive teaching-learning conversations.
Outcome:	The proposed workflow integrates teacher and learner agents, an interaction manager, and an evaluator to facilitate procedural learning and assess pedagogic quality.

Copied to clipboard

Haoxiang Sun, Ruize Gao, Pei Zhang, Baosong Yang, Rui Wang

Machine Translation

Challenge:	Current approaches to constructing preference data rely on human annotations.
Approach:	They propose a framework which efficiently constructs translation preference data for iterative training.
Outcome:	The proposed framework improves translation preference data on large language models.

Doc-V*: Coarse-to-Fine Interactive Visual Reasoning for Multi-Page Document VQA (2026.acl-long)

Copied to clipboard

Yuanlei Zheng, Pei Fu, Hang Li, Ziyang Wang, Yuyi Zhang, Wenyu Ruan, Xiaojin Zhang, Zhongyu Wei, Zhenbo Luo, Jian Luan, Wei Chen, Xiang Bai

Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization (2024.acl-long)

Challenge:	Existing OCR-free approaches to document visual question answering are brittle and passive.
Approach:	They propose an OCR-free agentic framework that casts multi-page DocVQA as sequential evidence aggregation.
Outcome:	The proposed framework outperforms open-source and proprietary models in five benchmarks and improves out-of-domain performance by 47.9% over baseline.

Copied to clipboard

Zhexin Zhang, Junxiao Yang, Pei Ke, Fei Mi, Hongning Wang, Minlie Huang

Robustness in NLP

Challenge:	Despite the success of jailbreaking attacks, there is a lack of effort in defending against them.
Approach:	They propose to integrate goal prioritization at both training and inference stages to counteract this conflict between the goals of being helpful and ensuring safety.
Outcome:	The proposed approach reduces the Attack Success Rate (ASR) of jailbreaking from 66.4% to 3.6% for ChatGPT and reduces it from 71.0% to 6.6% for Llama2-13B.

Autonomous Workflow for Multimodal Fine-Grained Training Assistants Towards Mixed Reality (2024.findings-acl)

Copied to clipboard

Jiahuan Pei, Irene Viola, Haochen Huang, Junxiao Wang, Moonisa Ahsan, Fanghua Ye, Jiang Yiming, Yao Sai, Di Wang, Zhumin Chen, Pengjie Ren, Pablo Cesar

HPSS: Heuristic Prompting Strategy Search for LLM Evaluators (2025.findings-acl)

Challenge:	a fine-grained, comprehensive understanding of multimodal environments remains under-explored.
Approach:	They propose an automated workflow for integrating AI agents into extended reality (XR) they propose a cerebral language agent that integrates LLM with memory, planning, and interaction with XR tools and a vision-language agent .
Outcome:	The proposed workflow integrates AI agents seamlessly into extended reality (XR) applications for fine-grained training.

Copied to clipboard

Bosi Wen, Pei Ke, Yufei Sun, Cunxiang Wang, Xiaotao Gu, Jinfeng Zhou, Jie Tang, Hongning Wang, Minlie Huang

CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation (2024.acl-long)

Challenge:	Existing efforts to optimize text evaluation prompts neglect the combinatorial impact of multiple factors, leading to insufficient optimization of the evaluation pipeline.
Approach:	They propose to integrate 8 key factors for evaluation prompts and integrate them into an algorithm that searches for well-behaved prompting strategies for LLM evaluators.
Outcome:	The proposed method outperforms existing methods and human-designed evaluation prompts on four evaluation tasks.

Copied to clipboard

Pei Ke, Bosi Wen, Andrew Feng, Xiao Liu, Xuanyu Lei, Jiale Cheng, Shengyuan Wang, Aohan Zeng, Yuxiao Dong, Hongning Wang, Jie Tang, Minlie Huang

CharacterGLM: Customizing Social Characters with Large Language Models (2024.emnlp-industry)

Challenge:	Existing models for NLP evaluations lack the ability to generate informative critiques in pointwise grading and pairwise comparison especially without references.
Approach:	They propose a method which can acquire pointwise grading critiques with pseudo references and revise these critiques via multi-path prompting to obtain informative evaluation data in different tasks and settings.
Outcome:	The proposed method outperforms all open-source models and even GPT-4 in system-level correlations of pointwise grading.

Copied to clipboard

Jinfeng Zhou, Zhuang Chen, Dazhen Wan, Bosi Wen, Yi Song, Jifan Yu, Yongkang Huang, Pei Ke, Guanqun Bi, Libiao Peng, JiaMing Yang, Xiyao Xiao, Sahand Sabour, Xiaohan Zhang, Wenjing Hou, Yijia Zhang, Yuxiao Dong, Hongning Wang, Jie Tang, Minlie Huang

EviReport: From Reasoned Outlines to Evidence Tracked Long-Form Reports (2026.findings-acl)

Challenge:	Character-based dialogue systems (CharacterDial) allow users to customize social characters for social interactions.
Approach:	They will collect a large-scale Chinese corpus of characters with diverse categories and behaviors and develop CharacterGLM models to address these challenges.
Outcome:	Experiments show that CharacterGLM outperforms most popular open- and closed-source LLMs and performs comparable to GPT-4.

Copied to clipboard

Zihan Liu, Jianhui li, Zexin Wang, Fei Sun, Jingjing LI, Zheyuan Li, Ke Xiang, Hang Cui, Houhua Gong, Changhua Pei, Gaogang Xie

CAML: A Conflict-Aware Molecular Language Model Merging Framework for Multi-Constraint Molecular Generation (2026.acl-long)

Challenge:	Evidence-intensive reports often produce fluent but under-supported drafts . eviReport is an evidence-grounded workflow for automated long-form report generation .
Approach:	They propose an evidence-tracked workflow that organizes corpus evidence into compact, traceable units and retrieves query-relevant subgraphs into retrieval-ready packages.
Outcome:	The proposed workflow outperforms baselines in factual coverage, factual accuracy and visual evidence integration.

Copied to clipboard

Xuanbai Ren, Luoda Tan, Pei Liu, Tengfei Ma, Xiangzheng Fu, Longyue Wang, Yiping Liu, Xiangxiang Zeng

Detecting Customer Complaint Escalation with Recurrent Neural Networks and Manually-Engineered Features (N19-2)

Challenge:	Existing paradigms struggle with this challenge due to catastrophic forgetting or gradient conflicts.
Approach:	They propose a conflict-aware molecular language model merging framework that generates multiple constraints moleculaire as a cooperative game among property-specific fine-tune models.
Outcome:	The proposed framework generates multiple constraints molecular as a cooperative game among property-specific fine-tune models (expert models) it minimizes conflicts among properties by exploring the optimal combination of the importance of the task parameter and relative fusion weights of each expert (fusion coefficient).

Copied to clipboard

Wei Yang, Luchen Tan, Chunwei Lu, Anqi Cui, Han Li, Xi Chen, Kun Xiong, Muzi Wang, Ming Li, Jian Pei, Jimmy Lin

Challenge:	e-commerce companies often have the option of escalating complaints by filing grievances with a government authority . this is detrimental to an ecommerce company, but this problem is challenging to solve by integrating recurrent neural networks with manually-engineered features.
Approach:	They propose a model that integrates recurrent neural networks with manually-engineered features to identify cases where the customer expresses such an intent.
Outcome:	The proposed model outperforms baseline models and provides better recall and triage for specialized agents.

Accelerate Parallelizable Reasoning via Parallel Decoding within One Sequence (2025.emnlp-main)

Copied to clipboard

Yijiong Yu, Wei Wang, Ran Chen, Ji Pei

Distribution Calibration for Out-of-Domain Detection with Bayesian Approximation (2022.coling-1)

Challenge:	Existing methods for parallelizable reasoning tasks are inefficient, says a new study . generating lengthy reasoning sequences is computationally expensive and time-consuming, says the study authors .
Approach:	They propose a method that decodes multiple tokens per forward pass using a tree-like attention mask . their method achieves nearly 100% speedup in decoding while basically maintaining the answer quality .
Outcome:	Experimental results show that the method achieves nearly 100% speedup in decoding while maintaining the answer quality.

Copied to clipboard

Yanan Wu, Zhiyuan Zeng, Keqing He, Yutao Mou, Pei Wang, Weiran Xu

Challenge:	Existing methods for OOD detection are overconfident for OD samples . lack of labeled OOD examples leads to poor prior knowledge about these unknown intents, making it challenging to detect OOD samples.
Approach:	They propose a Bayesian OOD detection framework to calibrate distribution uncertainty using Monte-Carlo Dropout.
Outcome:	The proposed framework gains 33.33% OOD F1 improvements with increasing only 0.41% inference time compared to previous methods.

WildFeedback: Aligning LLMs With In-situ User Interactions And Feedback (2026.acl-long)

Copied to clipboard

Taiwei Shi, Zhuoer Wang, Longqi Yang, Ying-Chun Lin, Zexue He, Mengting Wan, Pei Zhou, Sujay Kumar Jauhar, Sihao Chen, Shan Xia, Hongfei Zhang, Jieyu Zhao, Xiaofeng Xu, Xia Song, Jennifer Neville

Challenge:	Traditional alignment methods rely on human annotations and are subjective and misalignment with real-world user preferences.
Approach:	They propose a framework that leverages in-situ user feedback during conversations with LLMs to create preference datasets automatically.
Outcome:	The proposed framework identifies and classifies user feedback to LLM responses between conversation turns and creates examples of preferred and dispreferred responses according to user preferences.

Transferable Direct Prompt Injection via Activation-Guided MCMC Sampling (2025.emnlp-main)

Copied to clipboard

Minghui Li, Hao Zhang, Yechao Zhang, Wei Wan, Shengshan Hu, Pei Xiaobing, Jing Wang