Papers by Pei Chen

Multimodal Large Language Models for Text-rich Image Understanding: A Comprehensive Review (2025.findings-acl)

Challenge:	Existing preference learning methods rely heavily on curated data from humans or advanced LLMs, which is costly and difficult to scale.
Approach:	They propose a framework that leverages implicit preferences in unlabeled user-generated content to generate preference data.
Outcome:	The proposed framework transforms user-generated content into user queries and generates responses from the policy model.

Copied to clipboard

Pei Fu, Tongkun Guan, Zining Wang, Zhentao Guo, Chen Duan, Hao Sun, Boming Chen, Qianyi Jiang, Jiayao Ma, Kai Zhou, Junfeng Luo

Multimodality

Challenge:	Recent advances in vision-language models have unified perception and understanding tasks within Visual Question Answering paradigms.
Approach:	They propose to outline timeline, architecture, and pipeline of nearly all TIU MLLMs and review their performance on mainstream benchmarks.
Outcome:	The proposed models perform well on mainstream benchmarks and are compared with other models.

Mitigating Lost in Multi-turn Conversation via Curriculum RL with Verifiable Accuracy and Abstention Rewards (2026.acl-long)

Copied to clipboard

Ming Li, Pei Chen, Zhenhao Zhang, Tao Yang, Xinyang Zhang, Han Li, Tianyu Cao, Ming Zeng, Zhuofeng Wu, Meng Jiang, Huasheng Li, Lihong Li, Bing Yin

StoryMI: Steerable Multi-Agent Therapeutic Dialogue Generation (2026.findings-acl)

Challenge:	Large Language Models exhibit strong capabilities in single-turn instruction following but suffer from Lost-in-Conversation (LiC) when instructions are revealed progressively in multi-turn settings, models get "Lost in Conversation"
Approach:	They propose a framework that encourages models to generate correct answers and judge solvability in multi-turn conversations.
Outcome:	The proposed framework improves models' ability to balance problem-solving with abstention . it reduces premature answering behaviors that cause lost-in-conversation (LiC)

Copied to clipboard

Qingyu Meng, Min Chen, Dingming Liu, Yifan Mo, Yue Su, Xin Sun, Koen Hindriks, Jiahuan Pei

Syllogistic Reasoning for Legal Judgment Analysis (2023.emnlp-main)

Challenge:	Motivational interviewing (MI) is a directive, client-centered counseling approach for eliciting clients' motivation for behavioral change.
Approach:	They propose a multi-LLM agent framework for controllable MI dialogue generation . therapist and client agents generate MI-coded utterances guided by MI codes .
Outcome:	The proposed framework can generate fluent dialogues with minimal intervention time and a high level of evaluation.

Copied to clipboard

Wentao Deng, Jiahuan Pei, Keyi Kong, Zhe Chen, Furu Wei, Yujun Li, Zhaochun Ren, Zhumin Chen, Pengjie Ren

ALERT: An LLM-powered Benchmark for Automatic Evaluation of Recommendation Explanations (2025.naacl-long)

Challenge:	Legal judgment assistants are developing fast due to impressive progress of large language models.
Approach:	They construct and manually correct a syllogistic reasoning dataset for legal judgment analysis using large language models as benchmarks.
Outcome:	The proposed dataset contains 11,239 criminal cases covering 4 criminal elements, 80 charges and 124 articles.

Copied to clipboard

Yichuan Li, Xinyang Zhang, Chenwei Zhang, Mao Li, Tianyi Liu, Pei Chen, Yifan Gao, Kyumin Lee, Kaize Ding, Zhengyang Wang, Zhihan Zhang, Jingbo Shang, Xian Li, Trishul Chilimbi

EVM-QuestBench: An Execution-Grounded Benchmark for Natural-Language Transaction Code Generation (2026.acl-long)

Challenge:	Existing benchmarks for recommendation explanation evaluation lack item diversity and user preferences data.
Approach:	They propose a model-agnostic recommendation explanation evaluation benchmark based on Amazon e-commerce categories with implicit preferences . they propose two novel automatic evaluators that enable scalable and human-preference aligned evaluation of explanations .
Outcome:	The proposed model-agnostic evaluation benchmark outperforms existing methods in a variety of domains.

Copied to clipboard

Pei Yang, Wanyi Chen, Ke Wang, Lynn Ai, Eric Yang, Tianyu Shi

Programming Languages in NLP

Challenge:	Existing evaluations of large language models overlook execution accuracy and safety.
Approach:	They propose an execution-grounded benchmark for natural-language transaction-script generation on EVM-compatible chains.
Outcome:	The proposed benchmark finds large performance gaps in the models with 5 independent rounds.

Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training (2025.naacl-long)

Copied to clipboard

Yuchen Zhuang, Jingfeng Yang, Haoming Jiang, Xin Liu, Kewei Cheng, Sanket Lokegaonkar, Yifan Gao, Qing Ping, Tianyi Liu, Binxuan Huang, Zheng Li, Zhengyang Wang, Pei Chen, Ruijie Wang, Rongzhi Zhang, Nasser Zalmout, Priyanka Nigam, Bing Yin, Chao Zhang

Bridging the Gap between Language Models and Cross-Lingual Sequence Labeling (2022.naacl-main)

Challenge:	Existing LLMs often rely on complex prompting or extensive fine-tuning to introduce new capabilities while preserving strong generalizability.
Approach:	They propose a large-scale pre-training corpus to enhance LLM agents' capabilities . they use 103B agent-specific data encompassing 76,537 APIs .
Outcome:	The proposed training corpus outperforms open-source LLMs and commercial LLM agents on three agent benchmarks.

Copied to clipboard

Nuo Chen, Linjun Shou, Ming Gong, Jian Pei, Daxin Jiang

Intent-calibrated Self-training for Answer Selection in Open-domain Dialogues (2023.tacl-1)

Challenge:	Existing methods to train cross-lingual pre-trained language models have shown great success in cross-linguistic sequence labeling tasks.
Approach:	They propose a cross-lingual language informative span masking task to eliminate the objective gap between pre-training and fine-tuning stages.
Outcome:	The proposed method surpasses the state-of-the-art methods on multiple benchmarks even with limited pre-training data.

Copied to clipboard

Wentao Deng, Jiahuan Pei, Zhaochun Ren, Zhumin Chen, Pengjie Ren

Natural Language Interfaces

Challenge:	Existing answer selection models require large amounts of labeled data to produce accurate answers.
Approach:	They propose intent-calibrated self-training to calibrate answer labels using labeled data . they propose intentcalibration to improve quality of pseudo answer labels .
Outcome:	The proposed intent-calibrated answer selection paradigm outperforms baselines with 1%, 5%, and 10% labeled data on two benchmark datasets.

MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing (2026.acl-industry)

Copied to clipboard

Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wenzheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Zirui Tang, Boyu Niu, Yuanhong Zheng, Dongsheng Ma, Ziyang Miao, Hejun Dong, Siyi Qian, Junyuan Zhang, Fangdong Wang, Jingzhou Chen, Xiaomeng Zhao, Liqun Wei, Wei Li, Shasha Wang, RuiLiang Xu, Yuanyuan Cao, Lu Chen, Qianqian Wu, Huaiyu Gu, Lindong Lu, Dechen Lin, null Shenguanlin, Xuanhe Zhou, Linfeng Zhang, Yuhang Zang, Xiaoyi Dong, Jiaqi Wang, Bo Zhang, Lei Bai, Pei Chu, Weijia Li, Jiang Wu, Lijun Wu, Zhenxiang Li, Guangyu Wang, Zhongying Tu, Chao Xu, Kai Chen, Bowen Zhou, Dahua Lin, Wentao Zhang, Conghui He

Green & Sustainable NLP

Challenge:	Document images are characterized by higher resolutions, denser content, and more complex structural layouts.
Approach:	They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition.
Outcome:	The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead.

BiKT: Enabling Bidirectional Knowledge Transfer Between Pretrained Models and Sequential Downstream Tasks (2024.findings-emnlp)

Copied to clipboard

Hang Zeng, Chaoyue Niu, Fan Wu, Shaojie Tang, Leihao Pei, Chengfei Lv, Guihai Chen

Instant Personalized Large Language Model Adaptation via Hypernetwork (2026.acl-long)

Challenge:	Existing frameworks adapt from initial pretrained model to each downstream task directly, but ignore sequential nature of downstream tasks and feedback effect on pretrained models.
Approach:	They propose a framework to enable bidirectional knowledge transfer between pretrained models and downstream tasks in rounds.
Outcome:	The proposed framework improves on 9 GLUE datasets and 6 SuperGLUEs.

Copied to clipboard

Zhaoxuan Tan, Zixuan Zhang, Haoyang Wen, Zheng Li, Rongzhi Zhang, Pei Chen, Fengran Mo, Zheyuan Liu, Qingkai Zeng, Qingyu Yin, Meng Jiang

IEvoAgent: Evolving Conversational Agent based on User Implicit Feedback (2026.acl-long)

Challenge:	Existing parameter-efficient fine-tuning methods require training a separate adapter for each user, making them computationally expensive and impractical for real-time updates.
Approach:	They propose a scalable framework that maps a user's profile directly to a full set of adapter parameters.
Outcome:	The proposed framework outperforms prompt-based personalization and OPPU while using substantially fewer computational resources at deployment.

Copied to clipboard

Yichen Cai, Jiayang Li, Junyuan Qiu, Jingya Guo, Weitao You, Changyuan Yang, Lingyun Sun, Pei Chen

UniConv: Unifying Retrieval and Response Generation for Large Language Models in Conversations (2025.acl-long)

Challenge:	Existing approaches to optimize conversational agents often rely on explicit preference pairs and expert evaluations.
Approach:	They propose a conversational agent framework that leverages the structured dependency between agent responses and user reactions to extract implicit feedback.
Outcome:	The proposed framework improves on MT-Bench-101, WildBench, and FB-Bech, and shows that mining implicit feedback supports better multi-turn alignment under evolving user preferences.

Copied to clipboard

Fengran Mo, Yifan Gao, Chuan Meng, Xin Liu, Zhuofeng Wu, Kelong Mao, Zhengyang Wang, Pei Chen, Zheng Li, Xian Li, Bing Yin, Meng Jiang

ShopSimulator: Evaluating and Exploring RL-Driven LLM Agent for Shopping Assistants (2026.acl-long)

Challenge:	Existing conversational search systems are usually built with two different models . this separation restricts the system from leveraging the model's intrinsic knowledge simultaneously . Existing studies for developing unified models cannot fully address the aspects of understanding conversational context, managing retrieval independently, and generating responses.
Approach:	They propose to unify dense retrieval and response generation for large language models in conversation by fine-tuning and mitigating data discrepancy.
Outcome:	The proposed model can outperform existing models on five conversational search datasets and reduce inconsistency risks while mitigating data discrepancy.

Copied to clipboard

Pei Wang, Yanan Wu, Xiaoshuai Song, Weixun Wang, Gengru Chen, Zhongwen Li, Kezhong Yan, Qi Liu, Ken Deng, Shuaibing Zhao, Shaopan Xiong, Xuepeng Liu, Xuefeng Chen, Wanxi Deng, Wenbo Su, Bo Zheng

Challenge:	Existing studies on large language model-based agents focus on evaluation benchmarks without training support.
Approach:	They propose a large-scale Chinese shopping simulation environment that uses large language models to train agents.
Outcome:	The proposed model performs poorly in a large-scale and challenging shopping environment in China.

Efficient Domain Adaptation for Non-Autoregressive Machine Translation (2024.findings-acl)

Copied to clipboard

WangJie You, Pei Guo, Juntao Li, Kehai Chen, Min Zhang

Lattice Transformer for Speech Translation (P19-1)

Challenge:	Existing non-parametric approaches like nearest neighbor machine translation have made small Autoregressive translation models less efficient . despite their impressive generalization and task performance, LLMs suffer from prohibitive inference cost when confronted with specific domains.
Approach:	They propose a domain adaptation approach that tailors a k-nearest-neighbor algorithm for NAT models that incorporates the parallel nature of NAT.
Outcome:	The proposed approach achieves significant improvements over the Base-NAT model and exhibits enhanced efficiency.

Copied to clipboard

Pei Zhang, Niyu Ge, Boxing Chen, Kai Fan

Text Generation

Challenge:	Recent advances in sequence modeling have highlighted the strengths of the transformer architecture.
Approach:	They propose a general lattice transformer for speech translation where the input is the output of the automatic speech recognition (ASR) they propose 'controllable' lattica attention mechanism to consume latent representations.
Outcome:	The proposed model outperforms baseline and lattice LSTM on the Chinese-English translation task.

LongLeader: A Comprehensive Leaderboard for Large Language Models in Long-context Scenarios (2025.naacl-long)

Copied to clipboard

Pei Chen, Hongye Jin, Cheng-Che Lee, Rulin Shao, Jingfeng Yang, Mingyu Zhao, Zhaoyu Zhang, Qin Lu, Kaiwen Men, Ning Xie, Huasheng Li, Bing Yin, Han Li, Lingyun Wang

Trajectory2Task: Training Robust Tool-Calling Agents with Synthesized Yet Verifiable Data for Complex User Intents (2026.acl-long)

Challenge:	LongLeader aims to assess different LLMs' long-context comprehension abilities . long-constext comprehension is a key bottleneck for many use cases .
Approach:	They propose a leaderboard to assess different LLMs' long-context comprehension abilities . they offer open-source access to the benchmarks and maintain a dedicated website .
Outcome:	The proposed model assesses different LLMs on selected benchmarks and provides open-source access to the benchmarks.

Copied to clipboard

Ziyi Wang, Yuxuan Lu, Yimeng Zhang, Pei Chen, Ziwei Dong, Jing Huang, Jiri Gesi, Xianfeng Tang, Chen Luo, Qun Liu, Yisi Sang, Hanqing Lu, Manling Li, Jin Lai, Dakuo Wang

Improving Large Language Models Function Calling and Interpretability via Guided-Structured Templates (2025.emnlp-main)

Challenge:	Tool-calling agents are increasingly deployed in real-world customer-facing workflows . but most studies on tool-callers focus on idealized settings with general, fixed, and well-specified tasks.
Approach:	They propose a tool-calling agent-based data pipeline that converts trajectories into user-facing tasks with controlled intent adaptations.
Outcome:	The proposed pipeline can be used to study tool use under three scenarios.

Copied to clipboard

Hy Dang, Tianyi Liu, Zhuofeng Wu, Jingfeng Yang, Haoming Jiang, Tao Yang, Pei Chen, Zhengyang Wang, Helen Wang, Huasheng Li, Bing Yin, Meng Jiang

MELoRA: Mini-Ensemble Low-Rank Adapters for Parameter-Efficient Fine-Tuning (2024.acl-long)

Challenge:	Large language models (LLMs) have strong reasoning and tool-use capabilities, yet fail in real-world tool-interactions due to incorrect parameterization, poor tool selection, or misinterpretation of user intent.
Approach:	They propose a curriculum-inspired framework that leverages structured reasoning templates to guide LLMs through more deliberate step-by-step instructions for generating function calls.
Outcome:	The proposed framework reduces tool-use errors and improves interpretability and transparency of tool-using agents.

Copied to clipboard

Pengjie Ren, Chengshun Shi, Shiguang Wu, Mengqi Zhang, Zhaochun Ren, Maarten Rijke, Zhumin Chen, Jiahuan Pei

CoMM: Collaborative Multi-Agent, Multi-Reasoning-Path Prompting for Complex Problem Solving (2024.findings-naacl)

Challenge:	Large language models (LLMs) are the default paradigm for natural language processing (NLP) as the models’ scale and the diversity of tasks increase, fine-tuning becomes infeasible.
Approach:	They propose to freeze original pretrained weights and train a group of mini LoRAs with only a small number of parameters and reduce their rank by 8 times .
Outcome:	The proposed model uses fewer trainable parameters while maintaining a higher rank, thereby offering improved performance potential.

Copied to clipboard

Pei Chen, Shuai Zhang, Boran Han

Low-Resource NLP

Challenge:	Large Language Models (LLMs) have shown great ability in solving traditional natural language tasks and elementary reasoning tasks with appropriate prompting techniques.
Approach:	They propose a collaborative multi-agent, multi-reasoning-path prompting framework that prompts LLMs to play different roles in a problem-solving team and encourages different role-play agents to collaboratively solve the target task.
Outcome:	The proposed framework is applied to two college-level science problems over competitive baselines.

Mosaic-IT: Cost-Free Compositional Data Synthesis for Instruction Tuning (2025.findings-acl)

Copied to clipboard

Ming Li, Pei Chen, Chenguang Wang, Hongyu Zhao, Yijun Liang, YuPeng Hou, Fuxiao Liu, Tianyi Zhou

Long-Short Term Masking Transformer: A Simple but Effective Baseline for Document-level Neural Machine Translation (2020.emnlp-main)

Challenge:	Current instruction tuning relies on teacher models or human intervention to generate and refine the instructions and responses for training, which are costly, non-sustainable, and may lack diversity.
Approach:	They propose a human/model-free compositional data synthesis method that can create rich and diverse augmentations from existing instruction tuning data to enhance large language models.
Outcome:	The proposed method improves performance over benchmarks and reduces training costs by 80% compared with original instruction tuning.

Copied to clipboard

Pei Zhang, Boxing Chen, Niyu Ge, Kai Fan

Improving Multilingual Sign Language Translation with Automatically Clustered Language Family Information (2025.coling-main)

Challenge:	Recent advances in deep learning have led to significant improvement of document-level neural machine translation (NMT).
Approach:	They propose a long-short term masking self-attention on top of the standard transformer to capture the long-range dependence and reduce the propagation of errors.
Outcome:	The proposed model captures the long-range dependence and reduces errors on two publicly available document-level datasets.

Copied to clipboard

Ruiquan Zhang, Cong Hu, Pei Yu, Yidong Chen

Multilinguality

Challenge:	Recent research has focused on bilingual translation models, but multilingual sign language translation presents unique challenges due to the diversity of sign languages across nations.
Approach:	They propose a method that leverages sign language families to improve MSLT performance.
Outcome:	The proposed approach can achieve balance between translation accuracy and computational cost by regulating the number of language families.

Reconstructing Event Regions for Event Extraction via Graph Attention Networks (2020.aacl-main)

Copied to clipboard

Pei Chen, Hang Yang, Kang Liu, Ruihong Huang, Yubo Chen, Taifeng Wang, Jun Zhao

Alleviating Over-smoothing for Unsupervised Sentence Representation (2023.acl-long)

Challenge:	Existing approaches for event extraction focus on sentence-level event extraction, but they lack a broader view of the document context.
Approach:	They build graphs with candidate event filler extractions enriched by sentential embeddings as nodes and use graph attention networks to identify event regions in a document and aggregate event information.
Outcome:	The proposed method performs well on two languages and shows that it is faster than previous methods.

Copied to clipboard

Nuo Chen, Linjun Shou, Jian Pei, Ming Gong, Bowen Cao, Jianhui Chang, Jia Li, Daxin Jiang

Probing into the Root: A Dataset for Reason Extraction of Structural Events from Financial Documents (2021.eacl-main)

Challenge:	Existing approaches to learn better unsupervised sentence representations have been successful . over-smoothing problem in unsupervised sentences reduces the capacity of powerful PLMs .
Approach:	They propose a method to solve the over-smoothing problem in unsupervised sentence representations by combining negatives from PLMs intermediate layers.
Outcome:	The proposed method improves on different strong baselines on Semantic Textual Similarity and Transfer datasets.

Copied to clipboard

Pei Chen, Kang Liu, Yubo Chen, Taifeng Wang, Jun Zhao

LEAF: Large Language Diffusion Model for Time Series Forecasting (2025.findings-emnlp)

Challenge:	Existing methods for event reason extraction are far from resolving this problem.
Approach:	They propose a task to extract causal explanations from document-level texts . they use a dataset FinReason for evaluation to provide Reasons annotation for financial events .
Outcome:	The proposed task performs better than existing methods on a dataset of 8,794 documents, 12,861 financial events and 11,006 reason spans.

Copied to clipboard

Yuhang Pei, Tao Ren, Yifan Wang, Zhipeng Sun, Wei Ju, Chong Chen, Xian-Sheng Hua, Xiao Luo

Long-context Language Models Fail in Basic Retrieval Tasks Without Sufficient Reasoning Steps (2025.findings-emnlp)

Challenge:	Recent work has applied large language models (LLMs) into time series forecasting, but they lack an understanding of holistic temporal patterns with potential error accumulation.
Approach:	They propose a framework that marries Larg e Langu age Diffusion Model with time series forecasting (LEAF) they propose converting time series into tokens and adopting language diffusion models to capture temporal dependencies.
Outcome:	The proposed framework generates future predictions with a diffusion model from a holistic view.

Copied to clipboard

Yijiong Yu, Zhixiao Qi, Yongfeng Huang, Wei Wang, null Weifeng.liu, Ran Chen, Ji Pei

ItD: Large Language Models Can Teach Themselves Induction through Deduction (2024.acl-long)

Challenge:	despite their extensive context window, long-context language models fail in some basic cases . a recent study shows that long-cot methods are not necessary for long-constituency tasks .
Approach:	a new study evaluates long-context language models with a large context window . the authors propose a method that can be well addressed with arbitrary reasoning steps .
Outcome:	The proposed methods are well addressed with a sufficient number of reasoning steps, guided by specific CoT prompts.

Copied to clipboard

Wangtao Sun, Haotian Xu, Xuanqing Yu, Pei Chen, Shizhu He, Jun Zhao, Kang Liu

From Experts to Bases: Orthogonal Subspace Mixture for Continual Multimodal Instruction Tuning (2026.acl-long)

Challenge:	Recent studies have shown that Large Language Models (LLMs) have limited ability to conduct induction.
Approach:	They propose a framework to enable LLMs to teach themselves induction through deduction.
Outcome:	The proposed framework improves performance on two induction benchmarks and shows that it can be used to teach induction through deduction.

Copied to clipboard

Pei Chen, Xilai Wang, null Shiqixu, Zejian Li, Lingyun Sun

Decoupling Pseudo Label Disambiguation and Representation Learning for Generalized Intent Discovery (2023.acl-long)

Challenge:	Existing parameter-efficient approaches to multimodal Continual Instruction Tuning suffer from knowledge interference and inefficient capacity expansion, limiting scalability.
Approach:	They propose a framework for multimodal Continual instruction tuning that decomposes adaptation weights into a globally shared pool of orthonormal bases to capture task-invariant knowledge.
Outcome:	Experiments show that MoBLoRA outperforms state-of-the-art methods while maintaining superior parameter efficiency.

Copied to clipboard

Yutao Mou, Xiaoshuai Song, Keqing He, Chen Zeng, Pei Wang, Jingang Wang, Yunsen Xian, Weiran Xu

Examining Gender Bias in Languages with Grammatical Gender (D19-1)

Challenge:	Existing methods for generalized intent discovery lack pseudo label disambiguation and representation learning.
Approach:	They propose a prototype learning framework to decouple pseudo label disambiguation and representation learning.
Outcome:	The proposed method can decouple pseudo label disambiguation and representation learning.

Copied to clipboard

Pei Zhou, Weijia Shi, Jieyu Zhao, Kuan-Hao Huang, Muhao Chen, Ryan Cotterell, Kai-Wei Chang

Multilinguality

Challenge:	Existing studies on gender bias in word embeddings focus on English . however, these studies cannot be extended to languages with morphological agreement on gender .
Approach:	They propose new metrics to evaluate gender bias in word embeddings of English and Spanish . they extend existing approaches to mitigate gender bias while preserving original embeddables .
Outcome:	The proposed methods reduce gender bias while preserving the original embeddings.

Training Medical QA Models Based on Mixed Rewards from Multiple-Choice and Open-Ended Questions (2025.findings-emnlp)

Copied to clipboard

Yue Qiu, Yujan Ting, Pei Dong, Terrence Chen, Weijing Huang

Question Answering

Challenge:	Reinforcement learning (RL) for large language models typically requires clear reward signals, which are often unavailable for open-ended (OE) questions where answer evaluation is ambiguous without scalable expert labeling.
Approach:	They propose a mixed-data approach to training large language models with varying reward clarity . they combine Multiple-choice questions (MCQs) with OE questions for which they use simpler, potentially noisy rewards such as Jaccard similarity or LLM-based evaluators.
Outcome:	The mixed-data approach improves medical question-answering performance across model scales.

Structural Contrastive Pretraining for Cross-Lingual Comprehension (2023.findings-acl)

Copied to clipboard

Nuo Chen, Linjun Shou, Tengtao Song, Ming Gong, Jian Pei, Jianhui Chang, Daxin Jiang, Jia Li

Retrofitting Contextualized Word Embeddings with Paraphrases (D19-1)

Challenge:	Existing methods to train multilingual language models using pretraining tasks like mask language modeling have yielded promising results on a wide range of downstream tasks.
Approach:	They propose a new task to align the structural words in a parallel sentence, enhancing models’ ability to comprehend cross-lingual representations.
Outcome:	The proposed task improves model's ability to comprehend cross-lingual representations by increasing the frequency of negative pairings.

Copied to clipboard

Weijia Shi, Muhao Chen, Pei Zhou, Kai-Wei Chang

Representation Learning

Challenge:	Contextualized word embeddings can be useful for downstream applications, but they can be over-sensitive to contexts.
Approach:	They propose a method to retrofit contextualized word embeddings with paraphrases to minimize the variance of word representations on paraphrased contexts.
Outcome:	The proposed method improves on sentence classification and inference tasks.

Doc-V*: Coarse-to-Fine Interactive Visual Reasoning for Multi-Page Document VQA (2026.acl-long)

Copied to clipboard

Yuanlei Zheng, Pei Fu, Hang Li, Ziyang Wang, Yuyi Zhang, Wenyu Ruan, Xiaojin Zhang, Zhongyu Wei, Zhenbo Luo, Jian Luan, Wei Chen, Xiang Bai

Multimodality

Challenge:	Existing OCR-free approaches to document visual question answering are brittle and passive.
Approach:	They propose an OCR-free agentic framework that casts multi-page DocVQA as sequential evidence aggregation.
Outcome:	The proposed framework outperforms open-source and proprietary models in five benchmarks and improves out-of-domain performance by 47.9% over baseline.

Autonomous Workflow for Multimodal Fine-Grained Training Assistants Towards Mixed Reality (2024.findings-acl)

Copied to clipboard

Jiahuan Pei, Irene Viola, Haochen Huang, Junxiao Wang, Moonisa Ahsan, Fanghua Ye, Jiang Yiming, Yao Sai, Di Wang, Zhumin Chen, Pengjie Ren, Pablo Cesar

Multimodality

Challenge:	a fine-grained, comprehensive understanding of multimodal environments remains under-explored.
Approach:	They propose an automated workflow for integrating AI agents into extended reality (XR) they propose a cerebral language agent that integrates LLM with memory, planning, and interaction with XR tools and a vision-language agent .
Outcome:	The proposed workflow integrates AI agents seamlessly into extended reality (XR) applications for fine-grained training.

Crossroads, Buildings and Neighborhoods: A Dataset for Fine-grained Location Recognition (2022.naacl-main)

Copied to clipboard

Pei Chen, Haotian Xu, Cheng Zhang, Ruihong Huang

CharacterGLM: Customizing Social Characters with Large Language Models (2024.emnlp-industry)

Challenge:	Named Entity Recognition (NER) datasets annotate coarse-grained entities such as a continent, a country, or a city.
Approach:	They propose a dataset HarveyNER with fine-grained locations annotated in tweets that characterizes many complex and long location mentions in informal descriptions.
Outcome:	The proposed dataset outperforms existing systems on hard cases and improves on the heuristic curricula.

Copied to clipboard

Jinfeng Zhou, Zhuang Chen, Dazhen Wan, Bosi Wen, Yi Song, Jifan Yu, Yongkang Huang, Pei Ke, Guanqun Bi, Libiao Peng, JiaMing Yang, Xiyao Xiao, Sahand Sabour, Xiaohan Zhang, Wenjing Hou, Yijia Zhang, Yuxiao Dong, Hongning Wang, Jie Tang, Minlie Huang

Detecting Customer Complaint Escalation with Recurrent Neural Networks and Manually-Engineered Features (N19-2)

Challenge:	Character-based dialogue systems (CharacterDial) allow users to customize social characters for social interactions.
Approach:	They will collect a large-scale Chinese corpus of characters with diverse categories and behaviors and develop CharacterGLM models to address these challenges.
Outcome:	Experiments show that CharacterGLM outperforms most popular open- and closed-source LLMs and performs comparable to GPT-4.

Copied to clipboard

Wei Yang, Luchen Tan, Chunwei Lu, Anqi Cui, Han Li, Xi Chen, Kun Xiong, Muzi Wang, Ming Li, Jian Pei, Jimmy Lin

Challenge:	e-commerce companies often have the option of escalating complaints by filing grievances with a government authority . this is detrimental to an ecommerce company, but this problem is challenging to solve by integrating recurrent neural networks with manually-engineered features.
Approach:	They propose a model that integrates recurrent neural networks with manually-engineered features to identify cases where the customer expresses such an intent.
Outcome:	The proposed model outperforms baseline models and provides better recall and triage for specialized agents.

Explicitly Capturing Relations between Entity Mentions via Graph Neural Networks for Domain-specific Named Entity Recognition (2021.acl-short)

Copied to clipboard

Pei Chen, Haibo Ding, Jun Araki, Ruihong Huang

Accelerate Parallelizable Reasoning via Parallel Decoding within One Sequence (2025.emnlp-main)

Challenge:	Named entity recognition (NER) is well studied for the general domain, but the performance is still moderate for specialized domains.
Approach:	They propose to explicitly connect entity mentions based on global coreference relations and local dependency relations to build better entity mention representations.
Outcome:	The proposed system improves the NER performance even with a tiny amount of labeled data.

Copied to clipboard

Yijiong Yu, Wei Wang, Ran Chen, Ji Pei

Reasoning

Challenge:	Existing methods for parallelizable reasoning tasks are inefficient, says a new study . generating lengthy reasoning sequences is computationally expensive and time-consuming, says the study authors .
Approach:	They propose a method that decodes multiple tokens per forward pass using a tree-like attention mask . their method achieves nearly 100% speedup in decoding while basically maintaining the answer quality .
Outcome:	Experimental results show that the method achieves nearly 100% speedup in decoding while maintaining the answer quality.

Causally Modeling the Linguistic and Social Factors that Predict Email Response (2025.naacl-long)

Copied to clipboard

Yinuo Xu, Hong Chen, Sushrita Rakshit, Aparna Ananthasubramaniam, Omkar Yadav, Mingqian Zheng, Michael Jiang, Lechen Zhang, Bowen Yi, Kenan Alkiek, Abraham Israeli, Bangzhao Shu, Hua Shen, Jiaxin Pei, Haotian Zhang, Miriam Schirmer, David Jurgens

Challenge:	a key intent behind many emails is to get a reply from the recipient.
Approach:	They propose to model the intents, expectations, and responsiveness in email exchanges by using a dataset containing 1800 emails annotated with nuanced types of intents and expectations.
Outcome:	The proposed model is based on 1800 emails annotated with nuanced types of intents and expectations . it shows that social status, argumentation, and strength of social connection influence email response rates .

ROSE: An Intent-Centered Evaluation Metric for NL2SQL (2026.acl-long)

Copied to clipboard

Wenqi Pei, Shizheng Hou, Boyan Li, Chen Han, Zhichao Shi, Yuyu Luo

Programming Languages in NLP

Challenge:	Existing evaluation metrics for evaluating the effectiveness of Natural Language to SQL (NL2SQL) solutions are becoming unreliable due to its sensitiveness to syntactic variation and inconsistent consistency with ground-truth SQL.
Approach:	They propose an intent-centered metric that focuses on whether the predicted SQL answers the question, rather than consistency with the ground-truth SQL.
Outcome:	The proposed metric outperforms the next-best metric by nearly 24% on the expert-aligned validation set ROSE-VEC.

WildFeedback: Aligning LLMs With In-situ User Interactions And Feedback (2026.acl-long)

Copied to clipboard

Taiwei Shi, Zhuoer Wang, Longqi Yang, Ying-Chun Lin, Zexue He, Mengting Wan, Pei Zhou, Sujay Kumar Jauhar, Sihao Chen, Shan Xia, Hongfei Zhang, Jieyu Zhao, Xiaofeng Xu, Xia Song, Jennifer Neville

Challenge:	Traditional alignment methods rely on human annotations and are subjective and misalignment with real-world user preferences.
Approach:	They propose a framework that leverages in-situ user feedback during conversations with LLMs to create preference datasets automatically.
Outcome:	The proposed framework identifies and classifies user feedback to LLM responses between conversation turns and creates examples of preferred and dispreferred responses according to user preferences.

More than Minorities and Majorities: Understanding Multilateral Bias in Language Generation (2024.findings-acl)

Copied to clipboard

Jiaxu Zhao, Zijing Shi, Yitong Li, Yulong Pei, Ling Chen, Meng Fang, Mykola Pechenizkiy

T3: Tree-Autoencoder Constrained Adversarial Text Generation for Targeted Attack (2020.emnlp-main)

Challenge:	Existing studies on bias dataset construction and mitigation focus on one demographic group . in real-world applications, there are more than two demographic groups at risk of the same bias.
Approach:	They propose to analyze and reduce biases across multiple demographic groups using a multi-demographic bias dataset.
Outcome:	The proposed method can mitigate biases among multiple demographic groups effectively, the authors show .

Copied to clipboard

Boxin Wang, Hengzhi Pei, Boyuan Pan, Qian Chen, Shuohang Wang, Bo Li

Robustness in NLP

Challenge:	Existing adversarial examples can induce arbitrary errors to the target models, but they can be exploited to estimate robustness of NLP models.
Approach:	They propose a target-controllable adversarial attack framework T3 to handle adversarials . they use tree-based decoders to regularize the syntactic correctness of generated text .
Outcome:	The proposed framework can be used to estimate the robustness of NLP models.

Teaching Language Models To Gather Information Proactively (2025.findings-emnlp)

Copied to clipboard

Tenghao Huang, Sihao Chen, Muhao Chen, Jonathan May, Longqi Yang, Mengting Wan, Pei Zhou