Papers by Pei Yang
ThinkPersona: Thinking with Persona Graphs for Faithful Individualized Role-Playing (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models are increasingly utilized as role-playing agents to simulate personas in interactive settings. |
| Approach: | They propose a role-playing agent trained to explicitly ground responses in individual identity. |
| Outcome: | The proposed agent can generate persona-consistent responses in long-context dialogues while maintaining general instruction-following capabilities. |
SelfPrompt: Autonomously Evaluating LLM Robustness via Domain-Constrained Knowledge Guidelines and Refined Adversarial Prompts (2025.coling-main)
Copied to clipboard
| Challenge: | Existing frameworks for evaluating robustness of large language models rely on standardized benchmarks that can escalate costs and limit evaluations across domains. |
| Approach: | They propose a framework to evaluate the robustness of large language models using adversarial prompts and domain-constrained knowledge guidelines. |
| Outcome: | The proposed framework reduces dependency on conventional benchmarks and provides efficient evaluations in constrained domains. |
Context-Interactive Pre-Training for Document Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Document machine translation typically suffers from a lack of document-level bilingual data. |
| Approach: | They propose a document machine translation model that incorporates contextual information into the training signals by capturing cross-sentence dependency within the target document and cross sentence translation to make better use of contextual information. |
| Outcome: | The proposed model outperforms baselines on three benchmark datasets and significantly outperformed previous approaches. |
PREADD: Prefix-Adaptive Decoding for Controlled Text Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for controlling text generation rely on auxiliary expert models but require training data. |
| Approach: | They propose a flexible method for controlled text generation that uses a prefix-prepended prompt instead of an external model. |
| Outcome: | The proposed method outperforms auxiliary-expert methods on three tasks by 12% or more on main metrics for each task. |
Mitigating Lost in Multi-turn Conversation via Curriculum RL with Verifiable Accuracy and Abstention Rewards (2026.acl-long)
Copied to clipboard
Ming Li, Pei Chen, Zhenhao Zhang, Tao Yang, Xinyang Zhang, Han Li, Tianyu Cao, Ming Zeng, Zhuofeng Wu, Meng Jiang, Huasheng Li, Lihong Li, Bing Yin
| Challenge: | Large Language Models exhibit strong capabilities in single-turn instruction following but suffer from Lost-in-Conversation (LiC) when instructions are revealed progressively in multi-turn settings, models get "Lost in Conversation" |
| Approach: | They propose a framework that encourages models to generate correct answers and judge solvability in multi-turn conversations. |
| Outcome: | The proposed framework improves models' ability to balance problem-solving with abstention . it reduces premature answering behaviors that cause lost-in-conversation (LiC) |
EVM-QuestBench: An Execution-Grounded Benchmark for Natural-Language Transaction Code Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing evaluations of large language models overlook execution accuracy and safety. |
| Approach: | They propose an execution-grounded benchmark for natural-language transaction-script generation on EVM-compatible chains. |
| Outcome: | The proposed benchmark finds large performance gaps in the models with 5 independent rounds. |
Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training (2025.naacl-long)
Copied to clipboard
Yuchen Zhuang, Jingfeng Yang, Haoming Jiang, Xin Liu, Kewei Cheng, Sanket Lokegaonkar, Yifan Gao, Qing Ping, Tianyi Liu, Binxuan Huang, Zheng Li, Zhengyang Wang, Pei Chen, Ruijie Wang, Rongzhi Zhang, Nasser Zalmout, Priyanka Nigam, Bing Yin, Chao Zhang
| Challenge: | Existing LLMs often rely on complex prompting or extensive fine-tuning to introduce new capabilities while preserving strong generalizability. |
| Approach: | They propose a large-scale pre-training corpus to enhance LLM agents' capabilities . they use 103B agent-specific data encompassing 76,537 APIs . |
| Outcome: | The proposed training corpus outperforms open-source LLMs and commercial LLM agents on three agent benchmarks. |
Meta-Reasoning: Semantics-Symbol Deconstruction for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods rely on syntactically mapping natural languages to complete formal languages like Python and SQL. |
| Approach: | They propose to deconstruct reasoning-independent semantic information into generic symbolic representations, thereby efficiently capturing more generalized reasoning knowledge. |
| Outcome: | The proposed method improves in-context reasoning accuracy, learning efficiency, out-of-domain generalization, and output stability compared to the Chain-of thought technique. |
IEvoAgent: Evolving Conversational Agent based on User Implicit Feedback (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to optimize conversational agents often rely on explicit preference pairs and expert evaluations. |
| Approach: | They propose a conversational agent framework that leverages the structured dependency between agent responses and user reactions to extract implicit feedback. |
| Outcome: | The proposed framework improves on MT-Bench-101, WildBench, and FB-Bech, and shows that mining implicit feedback supports better multi-turn alignment under evolving user preferences. |
LongLeader: A Comprehensive Leaderboard for Large Language Models in Long-context Scenarios (2025.naacl-long)
Copied to clipboard
Pei Chen, Hongye Jin, Cheng-Che Lee, Rulin Shao, Jingfeng Yang, Mingyu Zhao, Zhaoyu Zhang, Qin Lu, Kaiwen Men, Ning Xie, Huasheng Li, Bing Yin, Han Li, Lingyun Wang
| Challenge: | LongLeader aims to assess different LLMs' long-context comprehension abilities . long-constext comprehension is a key bottleneck for many use cases . |
| Approach: | They propose a leaderboard to assess different LLMs' long-context comprehension abilities . they offer open-source access to the benchmarks and maintain a dedicated website . |
| Outcome: | The proposed model assesses different LLMs on selected benchmarks and provides open-source access to the benchmarks. |
Large Language Model for Multi-Domain Translation: Benchmarking and Domain CoT Fine-tuning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Achieving consistent high-quality machine translation across diverse domains remains a challenge due to limited and imbalanced parallel training data available in various domains. |
| Approach: | They propose a domain Chain of Thought technique that uses the multi-domain intelligence of LLMs to improve translation performance. |
| Outcome: | The proposed method achieves significant improvements in translation accuracy and domain robustness over traditional fine-tuning on a small dataset of four domains. |
AnyTrans: Translate AnyText in the Image with Large Scale Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in natural language processing and computer vision have made it possible to translate images with text in one language into equivalent images displaying that text translated into another language. |
| Approach: | They propose an all-encompassing framework for the task–In-Image Machine Translation (IIMT) that incorporates contextual cues from both textual and visual elements during translation. |
| Outcome: | The proposed framework can be constructed using open-source models and requires no training, making it highly accessible and expandable. |
ThinkAnswer Loss: Balancing Semantic Similarity and Exact Matching for LLM Reasoning Enhancement (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for knowledge distillation use Chain-of-Thought (CoT) and answer pairs, but they lack appropriate supervision signals. |
| Approach: | They propose a framework that decouples CoT and answer supervision . the framework applies semantic similarity constraints while maintaining strict literal matching for the answer . |
| Outcome: | The proposed framework decouples CoT and answer supervision while maintaining strict literal matching for the answer. |
Improving Large Language Models Function Calling and Interpretability via Guided-Structured Templates (2025.emnlp-main)
Copied to clipboard
Hy Dang, Tianyi Liu, Zhuofeng Wu, Jingfeng Yang, Haoming Jiang, Tao Yang, Pei Chen, Zhengyang Wang, Helen Wang, Huasheng Li, Bing Yin, Meng Jiang
| Challenge: | Large language models (LLMs) have strong reasoning and tool-use capabilities, yet fail in real-world tool-interactions due to incorrect parameterization, poor tool selection, or misinterpretation of user intent. |
| Approach: | They propose a curriculum-inspired framework that leverages structured reasoning templates to guide LLMs through more deliberate step-by-step instructions for generating function calls. |
| Outcome: | The proposed framework reduces tool-use errors and improves interpretability and transparency of tool-using agents. |
CultureSynth: A Hierarchical Taxonomy-Guided and Retrieval-Augmented Framework for Cultural Question-Answer Synthesis (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Cultural competence is defined as the ability to understand and adapt to multicultural contexts. |
| Approach: | They propose a framework that uses a hierarchical multilingual taxonomy and a Retrieval-Augmented Generation to synthesize culturally relevant question-answer pairs. |
| Outcome: | The proposed framework contains a hierarchical multilingual taxonomy covering 12 primary and 130 secondary topics and a Retrieval-Augmented Generation (RAG)-based methodology leveraging factual knowledge to synthesize culturally relevant question-answer pairs. |
A Systematic Assessment of Language Models with Linguistic Minimal Pairs in Chinese (2026.tacl-1)
Copied to clipboard
Yikang Liu, Yeting Shen, Hongao Zhu, Lilong Xu, Zhiheng Qian, Siyuan Song, Kejia Zhang, Jialong Tang, Pei Zhang, Baosong Yang, Rui Wang, Hai Hu
| Challenge: | Using sub-linear length normalized log-probabilities (SLLN-LP), we find unequal lengths of sentences in minimal pairs difficult for LMs even up to 32B parameters. |
| Approach: | They propose to use ZhoBLiMP as a linguistic minimal pair benchmark for Chinese language models to mitigate biases. |
| Outcome: | The proposed metric mitigates biases in Chinese language models with over 100 paradigms . Anaphor, Quantifiers, and Ellipsis are difficult for LMs even up to 32B parameters . |
Training Language Model to Critique for Better Refinement (2025.findings-acl)
Copied to clipboard
Tianshu Yu, Chao Xiang, Mingchuan Yang, Pei Ke, Bosi Wen, Cunxiang Wang, Jiale Cheng, Li Zhang, Xinyu Mu, Chuxiong Sun, Minlie Huang
| Challenge: | Large language models (LLMs) have remarkable evaluation and critique capabilities, providing insightful feedback and identifying flaws in various tasks. |
| Approach: | They propose a framework to train critic models using refinement signals to generate feedback loops where critiques guide the model in refining its responses. |
| Outcome: | The proposed framework outperforms traditional methods and open-source models in terms of critique quality and refinement outcomes. |
ProMediate: A Simulation Testbed for Evaluating Proactive Mediation in Multi-Party Negotiation (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly integrated into agentic frameworks to assist individual users in completing diverse tasks. |
| Approach: | They propose a simulation environment with a plug-and-play proactive AI mediator . they use a socio-cognitive evaluation framework to measure consensus changes, intervention latency, mediator effectiveness and intelligence. |
| Outcome: | The proposed model outperforms a generic baseline in multi-party negotiation scenarios while being 77% faster in response. |
Think Before You Speak: Explicitly Generating Implicit Commonsense Knowledge for Response Generation (2022.acl-long)
Copied to clipboard
Pei Zhou, Karthik Gopalakrishnan, Behnam Hedayatnia, Seokhwan Kim, Jay Pujara, Xiang Ren, Yang Liu, Dilek Hakkani-Tur
| Challenge: | Current neural response generation models generate responses directly, omitting unstated implicit knowledge. |
| Approach: | They propose a generative approach to externalize implicit commonsense knowledge and use it to generate responses. |
| Outcome: | Empirical results show that TBS models outperform end-to-end RG models on most automatic metrics and generate more informative, specific, and commonsense-following responses. |
PAD: A Robustness Enhancement Ensemble Method via Promoting Attention Diversity (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to enhance robustness of deep neural networks focus on perturbation . weak robustness is a problem for many types of adversarial attacks, authors say . |
| Approach: | They propose a lightweight framework for enhancing robustness by perturbing parameters of a model and diversifying adversarial example distributions among different models. |
| Outcome: | The proposed method can improve robustness against adversarial attacks while maintaining accuracy on clean data. |
Reconstructing Event Regions for Event Extraction via Graph Attention Networks (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing approaches for event extraction focus on sentence-level event extraction, but they lack a broader view of the document context. |
| Approach: | They build graphs with candidate event filler extractions enriched by sentential embeddings as nodes and use graph attention networks to identify event regions in a document and aggregate event information. |
| Outcome: | The proposed method performs well on two languages and shows that it is faster than previous methods. |
SAMem: State-Aware Memory as a Fine-Grained Memory for LLM Agents in Decision-Making (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing experiential memory approaches rely on task-level memory, but this lacks the situational alignment required for complex multi-step decision-making. |
| Approach: | They propose a new fine-grained memory paradigm that aligns memory retrieval with the current state instead of storing and reusing globally shared experiences. |
| Outcome: | Experiments on complex decision-making benchmarks show that the proposed state-aware memory outperforms existing experiential memory approaches and significantly improves task-solving efficiency. |
Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation (2020.coling-main)
Copied to clipboard
| Challenge: | Cross-lingual Machine Reading Comprehension (CLMRC) is a challenging problem due to the lack of large-scale annotated datasets in low-source languages, such as Arabic, Hindi, and Vietnamese. |
| Approach: | They propose a novel approach to augment cross-lingual machine reading comprehension by combining knowledge from multiple language branch models into a single model for all target languages. |
| Outcome: | Extensive experiments on two CLMRC benchmarks show the proposed method is effective and robust to data noises. |
Translationese-index: Using Likelihood Ratios for Graded and Generalizable Measurement of Translationese (2025.emnlp-main)
Copied to clipboard
Yikang Liu, Wanyang Zhang, Yiming Wang, Jialong Tang, Pei Zhang, Baosong Yang, Fei Huang, Rui Wang, Hai Hu
| Challenge: | Translationese is a linguistic property that is often introduced in the translation process that is different from those of original texts. |
| Approach: | They propose to use synthesized translations and translations in the wild to evaluate T-index's generalizability in cross-domain settings and its validity against human judgments. |
| Outcome: | The proposed measure can generalize to unseen genres, authors, and language pairs. |
Competency-Aware Neural Machine Translation: Can Machine Translation Know its Own Translation Quality? (2022.emnlp-main)
Copied to clipboard
| Challenge: | Neural machine translation models are often criticized for failures that happen without competency awareness. |
| Approach: | They propose a method that extends conventional NMT with a self-estimator to translate a source sentence and estimate its competency. |
| Outcome: | The proposed method performs on translation tasks intact and on quality estimation tasks better than existing methods. |
Enhancing Machine Translation with Self-Supervised Preference Data (2025.acl-long)
Copied to clipboard
| Challenge: | Current approaches to constructing preference data rely on human annotations. |
| Approach: | They propose a framework which efficiently constructs translation preference data for iterative training. |
| Outcome: | The proposed framework improves translation preference data on large language models. |
Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization (2024.acl-long)
Copied to clipboard
| Challenge: | Despite the success of jailbreaking attacks, there is a lack of effort in defending against them. |
| Approach: | They propose to integrate goal prioritization at both training and inference stages to counteract this conflict between the goals of being helpful and ensuring safety. |
| Outcome: | The proposed approach reduces the Attack Success Rate (ASR) of jailbreaking from 66.4% to 3.6% for ChatGPT and reduces it from 71.0% to 6.6% for Llama2-13B. |
Socratic Style Chain-of-Thoughts Help LLMs to be a Better Reasoner (2025.findings-acl)
Copied to clipboard
| Challenge: | Synthetic data generation is a promising approach to enhance reasoning capabilities of large language models. |
| Approach: | They propose a multi-agent debate framework based on the Socratic questioning strategy . they use socratic questions to deepen the thinking process and broaden it to motivate self-reflection . |
| Outcome: | The proposed framework outperforms existing methods on math and code generation tasks while maintaining affordable costs. |
CharacterGLM: Customizing Social Characters with Large Language Models (2024.emnlp-industry)
Copied to clipboard
Jinfeng Zhou, Zhuang Chen, Dazhen Wan, Bosi Wen, Yi Song, Jifan Yu, Yongkang Huang, Pei Ke, Guanqun Bi, Libiao Peng, JiaMing Yang, Xiyao Xiao, Sahand Sabour, Xiaohan Zhang, Wenjing Hou, Yijia Zhang, Yuxiao Dong, Hongning Wang, Jie Tang, Minlie Huang
| Challenge: | Character-based dialogue systems (CharacterDial) allow users to customize social characters for social interactions. |
| Approach: | They will collect a large-scale Chinese corpus of characters with diverse categories and behaviors and develop CharacterGLM models to address these challenges. |
| Outcome: | Experiments show that CharacterGLM outperforms most popular open- and closed-source LLMs and performs comparable to GPT-4. |
Detecting Customer Complaint Escalation with Recurrent Neural Networks and Manually-Engineered Features (N19-2)
Copied to clipboard
Wei Yang, Luchen Tan, Chunwei Lu, Anqi Cui, Han Li, Xi Chen, Kun Xiong, Muzi Wang, Ming Li, Jian Pei, Jimmy Lin
| Challenge: | e-commerce companies often have the option of escalating complaints by filing grievances with a government authority . this is detrimental to an ecommerce company, but this problem is challenging to solve by integrating recurrent neural networks with manually-engineered features. |
| Approach: | They propose a model that integrates recurrent neural networks with manually-engineered features to identify cases where the customer expresses such an intent. |
| Outcome: | The proposed model outperforms baseline models and provides better recall and triage for specialized agents. |
WildFeedback: Aligning LLMs With In-situ User Interactions And Feedback (2026.acl-long)
Copied to clipboard
Taiwei Shi, Zhuoer Wang, Longqi Yang, Ying-Chun Lin, Zexue He, Mengting Wan, Pei Zhou, Sujay Kumar Jauhar, Sihao Chen, Shan Xia, Hongfei Zhang, Jieyu Zhao, Xiaofeng Xu, Xia Song, Jennifer Neville
| Challenge: | Traditional alignment methods rely on human annotations and are subjective and misalignment with real-world user preferences. |
| Approach: | They propose a framework that leverages in-situ user feedback during conversations with LLMs to create preference datasets automatically. |
| Outcome: | The proposed framework identifies and classifies user feedback to LLM responses between conversation turns and creates examples of preferred and dispreferred responses according to user preferences. |
Multi-Modal Sarcasm Detection via Cross-Modal Graph Convolutional Network (2022.acl-long)
Copied to clipboard
| Challenge: | Existing studies on multimodal sarcasm detection using textual and visual information have been limited to text-only approaches. |
| Approach: | They propose to construct a cross-modal graph for each multi-modal instance to explicitly draw the ironic relations between textual and visual modalities. |
| Outcome: | The proposed model achieves state-of-the-art in multi-modal sarcasm detection. |
GenTool: Enhancing Tool Generalization in Language Models through Zero-to-One and Weak-to-Strong Simulation (2025.findings-acl)
Copied to clipboard
Jie He, Jennifer Neville, Mengting Wan, Longqi Yang, Hui Liu, Xiaofeng Xu, Xia Song, Jeff Z. Pan, Pei Zhou
| Challenge: | Large Language Models (LLMs) can expand their capabilities by integrating external tools. |
| Approach: | They propose a training framework that prepares LLMs for diverse generalization challenges in tool utilization. |
| Outcome: | The proposed framework improves the tool-usage capabilities of LLMs by up to 8B parameters, surpassing GPT-4o. |
Locate-and-Focus: Enhancing Terminology Translation in Speech Language Models (2025.acl-long)
Copied to clipboard
Suhang Wu, Jialong Tang, Chengyi Yang, Pei Zhang, Baosong Yang, Junhui Li, Junfeng Yao, Min Zhang, Jinsong Su
| Challenge: | Existing methods for terminology translation struggle with interference from irrelevant noise. |
| Approach: | They propose a Locate-and-Focus method that locates terminologies within utterances to construct translation knowledge by minimizing irrelevant information for ST models. |
| Outcome: | The proposed method locates terminologies within utterances and enhances the success rate of terminology translation while maintaining robust general translation performance. |
Teaching Language Models To Gather Information Proactively (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models are often defaulted to passive responses or narrow clarifications when faced with incomplete or under-specified prompts. |
| Approach: | They propose a new task paradigm where LLMs must identify gaps in context and strategically elicit implicit user knowledge through targeted questions. |
| Outcome: | The proposed framework outperforms o3-mini on evaluation metrics and human annotators favor clarification questions and final outlines. |