Papers by Rui Ye

P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chains (2024.findings-emnlp)

Challenge:	Current methods rely on ranking losses to teach reward model to assess preferences, but they are susceptible to noise and ambiguous data, often failing to deeply understand human intentions.
Approach:	They propose a method that incorporates contrastive learning into the reward modeling process to enhance generalization and stabilize the reinforcement learning training process.
Outcome:	The proposed method enhances generalization of the reward model, stabilizes the reinforcement learning training process, and improves the final alignment with human preferences.

Copied to clipboard

Simeng Han, Aaron Yu, Rui Shen, Zhenting Qi, Martin Riddell, Wenfei Zhou, Yujie Qiao, Yilun Zhao, Semih Yavuz, Ye Liu, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Dragomir Radev, Rex Ying, Arman Cohan

Traffic Light or Light Traffic? Investigating Phrasal Semantics in Large Language Models (2024.findings-emnlp)

Challenge:	Existing methods on understanding the capabilities of LLMs in logical reasoning rely on binary entailment classification or synthetically derived rationales.
Approach:	They propose to annotate a human-annotated dataset consisting of diverse and complex reasoning chains for a set of realistic logical reasoning stories also written by humans.
Outcome:	The proposed model outperforms existing methods on understanding the capabilities of LLMs in logical reasoning by 10% or more.

Copied to clipboard

Rui Meng, Ye Liu, Lifu Tu, Daqing He, Yingbo Zhou, Semih Yavuz

MMAC: A Multilingual, Multimodal Alignment Framework for Cultural Grounding Evaluation (2026.acl-long)

Challenge:	Phrases are fundamental linguistic units through which humans convey semantics.
Approach:	They assess the capacity of API-based large language models to comprehend phrase semantics . they use three human-annotated datasets to analyze their results .
Outcome:	The proposed model outperforms embedding-based methods in phrase semantic reasoning tasks . the proposed model does not show significant advantage over fine-tuned methods .

Copied to clipboard

Weihua Zheng, Zhengyuan Liu, Tanmoy Chakraborty, Weiwen Xu, Xiaoxue Gao, Bryan Chen Zhengyu Tan, Bowei Zou, Chang Liu, Yujia Hu, Xing Xie, Xiaoyuan Yi, Jing Yao, Chaojun Wang, Long Li, Rui Liu, Huiyao Liu, Koji Inoue, Ryuichi Sumida, Tatsuya Kawahara, Fan Xu, Lingyu Ye, Wei Tian, Dongjun Kim, Jimin Jung, Jaehyung Seo, Nadya Yuki Wangsajaya, Pham Minh Duc, Ojasva Saxena, Palash Nandi, Xiyan Tao, Wiwik Karlina, Tuan Luong, Keertana Arun Vasan, Roy Ka-Wei Lee, Nancy F. Chen

Multimodality

Challenge:	Existing models lack cultural alignment across modalities and languages . a new framework to assess cultural awareness across linguistics and languages is needed .
Approach:	They propose a framework that integrates tri-modally aligned cultural benchmarks and a five-dimensional evaluation protocol to assess cross-country awareness disparities.
Outcome:	The proposed framework assesses cultural awareness disparities across modalities and languages . it is the first dataset aligned at the input level across text, image, and speech .

TrendSim: Simulating Trending Topics in Social Media Under Poisoning Attacks with LLM-based Multi-agent System (2025.findings-naacl)

Copied to clipboard

Zeyu Zhang, Jianxun Lian, Chen Ma, Yaning Qu, Ye Luo, Lei Wang, Rui Li, Xu Chen, Yankai Lin, Le Wu, Xing Xie, Ji-Rong Wen

Robustness in NLP

Challenge:	Trending topics bring in a new channel for poisoning attacks, resulting in negative impacts on society.
Approach:	They propose an LLM-based multi-agent system to simulate trending topics in social media . they propose a time-aware interaction mechanism, centralized message dissemination, and an interactive system .
Outcome:	The proposed system simulates trending topics under poisoning attacks on social media platforms.

DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AI (2024.findings-eacl)

Copied to clipboard

Jianguo Zhang, Kun Qian, Zhiwei Liu, Shelby Heinecke, Rui Meng, Ye Liu, Zhou Yu, Huan Wang, Silvio Savarese, Caiming Xiong

Dialogue Systems & Conversational Agents

Challenge:	DialogStudio is the largest and most diverse collection of dialogue datasets . existing datasets lack diversity and comprehensiveness, authors say .
Approach:	They introduce DialogStudio: the largest and most diverse collection of dialogue datasets . DialogStuio aggregates more than 80 diverse dialogue dataset .
Outcome:	a new dataset is created to improve the quality and diversity of dialogue datasets . DialogStudio is the largest and most diverse collection of dialogue data .

FedMABench: Benchmarking Mobile GUI Agents on Decentralized Heterogeneous User Data (2025.emnlp-main)

Copied to clipboard

WenHao Wang, Zijie Yu, Rui Ye, Jianqing Zhang, Guangyi Liu, Liang Liu, Siheng Chen, Yanfeng Wang

Challenge:	Mobile GUI agents have attracted tremendous research participation recently. traditional approaches to mobile agent training rely on centralized data collection.
Approach:	They propose a benchmark for federated training and evaluation of mobile GUI agents . they find that federation algorithms consistently outperform local training .
Outcome:	The first benchmark for federated training and evaluation of mobile GUI agents is released . it features 6 datasets with 30+ subsets, 8 federation algorithms, 10+ base models, and over 800 apps across 5 categories .

RethinkingTMSC: An Empirical Study for Target-Oriented Multimodal Sentiment Classification (2023.findings-emnlp)

Copied to clipboard

Junjie Ye, Jie Zhou, Junfeng Tian, Rui Wang, Qi Zhang, Tao Gui, Xuanjing Huang

Information Retrieval

Challenge:	Recent studies have shown that current TMSC systems rely on textual information, and the progress in tackling this task has slowed down.
Approach:	They propose to integrate both visual and textual information to improve the performance of TMSC by considering multimodal information.
Outcome:	The proposed model integrates both visual and textual information to improve performance.

Graph Enhanced Dual Attention Network for Document-Level Relation Extraction (2020.coling-main)

Copied to clipboard

Bo Li, Wei Ye, Zhonghao Sheng, Rui Xie, Xiangyu Xi, Shikun Zhang

SwapMoE: Serving Off-the-shelf MoE-based Large Language Models with Tunable Memory Budget (2024.acl-long)

Challenge:	Document-level relation extraction requires inter-sentence reasoning capabilities to capture local and global contextual information for multiple relation facts.
Approach:	They propose to characterize the interaction between sentences and potential relation instances via a Graph Enhanced Dual Attention network (GEDA) . they also propose a simple yet effective regularizer based on the natural duality of the S2R and R2S attentions, whose weights are also supervised by the supporting evidence of relation instances during training.
Outcome:	The proposed model achieves competitive performance on an existing large-scale dataset while the predictions can be interpretable and easily observed.

Copied to clipboard

Rui Kong, Yuanchun Li, Qingtian Feng, Weijun Wang, Xiaozhou Ye, Ye Ouyang, Linghe Kong, Yunxin Liu

Modeling discourse cohesion for discourse parsing via memory network (P18-2)

Challenge:	Mixture of experts (MoE) is a popular technique to improve capacity of Large Language Models (LLMs) but memory-constrained devices are a major concern in edge AI training and serving.
Approach:	They propose a framework for efficient serving of MoE-based large language models with tunable memory budgets.
Outcome:	Experiments show that SwapMoE can reduce memory consumption while maintaining reasonable accuracy.

Copied to clipboard

Yanyan Jia, Yuan Ye, Yansong Feng, Yuxuan Lai, Rui Yan, Dongyan Zhao

PURE: Aligning LLM via Pluggable Query Reformulation for Enhanced Helpfulness (2024.findings-emnlp)

Challenge:	Existing approaches to discourse parsing focus on studying the semantic and syntactic aspects of EDU pairs, but they do not address long span dependencies.
Approach:	They propose a new transition-based discourse parser that takes discourse cohesion into account by using memory networks.
Outcome:	The proposed method outperforms traditional features and improves performance on the RST discourse treebank.

Copied to clipboard

Wenjin Yao, Yidong Wang, Zhuohao Yu, Rui Xie, Shikun Zhang, Wei Ye

CPO: Addressing Reward Ambiguity in Role-playing Dialogue via Comparative Policy Optimization (2025.findings-emnlp)

Challenge:	Large language models (LLMs) depend on vast amounts of text data sourced from the Internet for their training.
Approach:	They propose a new alignment paradigm that reformulates risky queries into highly relevant yet harmless ones before feeding them into LLMs.
Outcome:	The proposed approach eliminates the high costs of training base LLMs and achieves a promising balance of harmlessness and helpfulness.

Copied to clipboard

Jing Ye, Rui Wang, Yuchuan Wu, Victor Ma, Feiteng Fang, Fei Huang, Yongbin Li

Dialogue Systems & Conversational Agents

Challenge:	Comparative Policy Optimization (CPO) redefines the reward evaluation paradigm by shifting from sample-wise scoring to comparative group-wise score.
Approach:	They propose a method to optimize subjective tasks by shifting from sample-wise to comparative group-wise scoring.
Outcome:	The proposed framework shifts from sample-wise scoring to comparative group-wise score . it minimizes contextual bias and enables more robust and fair performance evaluation.

On the Step Length Confounding in LLM Reasoning Data Selection (2026.findings-acl)

Copied to clipboard

Bing Wang, Rui Miao, Chen Shen, Shaotian Yan, Kaiyuan Liu, Ximing Li, Xiaosong Yuan, Sinan Fan, Jun Zhang, Jieping Ye

Reasoning

Challenge:	Existing pipelines generate long reasoning data from more capable Large Language Models (LLMs) and apply manually heuristic or naturalness-based selection methods to filter high-quality samples.
Approach:	They propose to use supervised fine-tuning to generate long reasoning data from more capable Large Language Models and apply manually heuristic or naturalness-based selection methods to filter high-quality samples.
Outcome:	Experiments on four LLMs and five evaluation benchmarks show that the proposed approach is effective in mitigating step length confounding problem.

PolicyLLM: Towards Excellent Comprehension of Public Policy for Large Language Models (2026.findings-acl)

Copied to clipboard

Han Bao, Penghao Zhang, Yue Huang, Zhengqing Yuan, Yanchi Ru, SU Rui, Yujun Zhou, Xiangqi Wang, Kehan Guo, Nitesh V Chawla, Yanfang Ye, Xiangliang Zhang

Uni-Parser: Unified Semantic Parser for Question Answering on Knowledge Base and Database (2022.emnlp-main)

Challenge:	Large Language Models (LLMs) are increasingly integrated into real-world decision-making, but their ability to comprehend and reason about policy-related content remains underexplored.
Approach:	They propose a bilingual benchmark evaluating policy comprehension comprising 21K cases across a broad spectrum of policy areas.
Outcome:	The proposed model shows stronger performance on application-oriented policy tasks than on memorization or conceptual understanding, and yields the highest accuracy on structured reasoning tasks.

Copied to clipboard

Ye Liu, Semih Yavuz, Rui Meng, Dragomir Radev, Caiming Xiong, Yingbo Zhou

Question Answering

Challenge:	Existing approaches on semantic parsing suffer from exponential growth of logical form candidates and can hardly generalize to unseen data.
Approach:	They propose a unified semantic parser for question answering on KB and DB . they define the primitive as the essential element in their framework .
Outcome:	The proposed framework can predict logical forms by altering and composing top-ranked primitives with different operations.

KnowledgeSG: Privacy-Preserving Synthetic Text Generation with Knowledge Distillation from Server (2024.emnlp-main)

Copied to clipboard

WenHao Wang, Xiaoyu Liang, Rui Ye, Jingyi Chai, Siheng Chen, Yanfeng Wang

Responsible & Trustworthy NLP

Challenge:	Existing methods to train large language models on private data are not effective because they rely on a local model for generation, resulting in a performance decline, or expose private data to API servers.
Approach:	They propose a client-server framework which enhances synthetic data quality and improves model performance while ensuring privacy.
Outcome:	The proposed framework improves model performance and privacy while learning local knowledge from the private data with differential privacy (DP) and distilling professional knowledge from server.

Exploiting Pseudo Image Captions for Multimodal Summarization (2023.findings-acl)

Copied to clipboard

Chaoya Jiang, Rui Xie, Wei Ye, Jinan Sun, Shikun Zhang

Text Generation

Challenge:	Existing approaches to multimodal summarization with multimodal output (MSMO) lack reference images for training, and exposure of image captions during training is inconsistent with MSMO’s task settings.
Approach:	They propose a coarse-to-fine image-text alignment mechanism to identify the most relevant sentence of each image in a document, resembling the role of image captions in capturing visual knowledge.
Outcome:	The proposed method sets up state-of-the-art on all intermodality and intramodality metrics and improves on image recommendation precision.

Enhancing In-Context Learning via Implicit Demonstration Augmentation (2024.acl-long)

Copied to clipboard

Xiaoling Zhou, Wei Ye, Yidong Wang, Chaoya Jiang, Zhemg Lee, Rui Xie, Shikun Zhang

UniRAG: Unified Query Understanding Method for Retrieval Augmented Generation (2025.acl-long)

Challenge:	In-context learning (ICL) is a new paradigm for pre-trained language models that can make predictions for unseen inputs without updating parameters.
Approach:	They propose a method that enables a model to augmented copies of a demonstration by leveraging their deep feature distribution and a logit calibration mechanism.
Outcome:	The proposed method significantly improves the average and worst-case accuracy across diverse PLMs and tasks.

Copied to clipboard

Rui Li, Liyang He, Qi Liu, Zheng Zhang, Heng Yu, Yuyang Ye, Linbo Zhu, Yu Su

Information Retrieval

Challenge:	Existing query augmentation methods face knowledge update lag and hallucinations in large language models (LLMs) Existing methods face two key challenges: (1) separation of query augmented and encoding tasks, which hinders information sharing and introduces cumulative errors; (2) difficulty of selecting optimal augmentation strategy for different scenarios.
Approach:	They propose a unified framework for query understanding in RAG that integrates internal and external knowledge to enhance query augmentation and encoding tasks.
Outcome:	The proposed framework outperforms traditional query augmentation methods in five knowledge-intensive benchmark tasks in both closed and open domain question answering.

Exploiting Entity BIO Tag Embeddings and Multi-task Learning for Relation Extraction with Imbalanced Data (P19-1)

Copied to clipboard

Wei Ye, Bo Li, Rui Xie, Zhonghao Sheng, Long Chen, Shikun Zhang

Synthesizing Post-Training Data for LLMs through Multi-Agent Simulation (2025.acl-long)

Challenge:	Existing methods to perform relation extraction are feature-based or kernel-based, but the results of our study show that they can improve the performance of a baseline model with more than 10% absolute increase in F1-score.
Approach:	They propose a multi-task architecture which jointly trains a model to perform relation identification with cross-entropy loss and relation classification with ranking loss.
Outcome:	The proposed model outperforms the state-of-the-art models on ACE 2005 Chinese and English corpus and significantly improves the performance of a baseline model with more than 10% increase in F1-score.

Copied to clipboard

Shuo Tang, Xianghe Pang, Zexi Liu, Bohan Tang, Rui Ye, Tian Jin, Xiaowen Dong, Yanfeng Wang, Siheng Chen

Challenge:	Extensive experiments demonstrate that our framework effectively generates both general and domain-specific data.
Approach:	They propose a multi-agent simulator that automatically generates diverse text-based scenarios, capturing a wide range of real-world human needs.
Outcome:	Experiments show that the proposed model outperforms Meta’s Llama-3-8B-Instruct model on AlpacaEval 2 and Arena-Hard benchmarks with just 20K instruction-response pairs.

RealBehavior: A Framework for Faithfully Characterizing Foundation Models’ Human-like Behavior Mechanisms (2023.findings-emnlp)

Copied to clipboard

Enyu Zhou, Rui Zheng, Zhiheng Xi, Songyang Gao, Xiaoran Fan, Zichu Fei, Jingting Ye, Tao Gui, Qi Zhang, Xuanjing Huang

Challenge:	Existing studies on human-like behaviors in foundation models do not verify their faithfulness . a simple application of psychological tools cannot faithfully characterize all human-type behaviors .
Approach:	They propose a framework to characterize humanoid behaviors in foundation models . they argue that a simple application of psychological tools cannot faithfully characterize all human-like behaviors .
Outcome:	The proposed framework assesses the faithfulness of results based on reproducibility, internal consistency, and generalizability.

BrowseConf: Confidence-Guided Test-Time Scaling for Web Agents (2026.findings-acl)

Copied to clipboard

Litu Ou, Kuan Li, Huifeng Yin, Liwen Zhang, Zhongwang Zhang, Xixi Wu, Rui Ye, Zile Qiao, Yong Jiang, Pengjun Xie, Fei Huang, Jingren Zhou

Challenge:	Existing work on confidence in LLMs is limited.
Approach:	They propose to use confidence scores to determine model answer quality and encourage model to try again until it reaches satisfactory confidence level.
Outcome:	The proposed methods significantly reduce token consumption while demonstrating competitive performance compared to baseline fixed budget methods.

HPE: Answering Complex Questions over Text by Hybrid Question Parsing and Execution (2023.findings-emnlp)

Copied to clipboard

Ye Liu, Semih Yavuz, Rui Meng, Dragomir Radev, Caiming Xiong, Shafiq Joty, Yingbo Zhou

Question Answering

Challenge:	End-to-end neural networks excel at answering natural language questions but fail on complex ones . a proposed framework for question parsing and execution on textual QA is designed to combine the strengths of neural and symbolic methods.
Approach:	They propose a framework for question parsing and execution on textual QA . they parse questions into an intermediate representation and use deterministic rules to translate them .
Outcome:	The proposed framework outperforms existing methods in supervised, few-shot, and zero-shot settings while preserving its underlying reasoning process.

FedDQC: Data Quality Control in Federated Instruction-tuning of Large Language Models (2025.findings-acl)

Copied to clipboard

Yaxin Du, Rui Ye, Fengting Yuchi, Wanru Zhao, Jingjing Qu, Yanfeng Wang, Siheng Chen

TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021.acl-demo)

Challenge:	Federated Learning (FL) enables privacy-preserving collaborative instruction tuning of large language models.
Approach:	They propose a federated instruction tuning framework with dynamic data quality control to solve this problem.
Outcome:	The proposed framework improves performance on mixed-quality datasets on synthetic and real-world datasets.

Copied to clipboard

Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang, Rui Zheng, Zexiong Pang, Qinzhuo Wu, Zhengyan Li, Chong Zhang, Ruotian Ma, Zichu Fei, Ruijian Cai, Jun Zhao, Xingwu Hu, Zhiheng Yan, Yiding Tan, Yuan Hu, Qiyuan Bian, Zhihua Liu, Shan Qin, Bolin Zhu, Xiaoyu Xing, Jinlan Fu, Yue Zhang, Minlong Peng, Xiaoqing Zheng, Yaqian Zhou, Zhongyu Wei, Xipeng Qiu, Xuanjing Huang

Robustness in NLP

Challenge:	Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction.
Approach:	They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack.
Outcome:	The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses.

Knowledge-guided Open Attribute Value Extraction with Reinforcement Learning (2020.emnlp-main)

Copied to clipboard

Ye Liu, Sheng Zhang, Rui Song, Suo Feng, Yanghua Xiao

Neural Topic Modeling with Bidirectional Adversarial Training (2020.acl-main)

Challenge:	Existing methods for open attribute value extraction for emerging entities are noisy or incomplete, even missing.
Approach:	They propose a knowledge-guided reinforcement learning framework for open attribute value extraction for emerging entities.
Outcome:	The proposed framework outperforms baselines by 16.5 - 27.8%.

Copied to clipboard

Rui Wang, Xuemeng Hu, Deyu Zhou, Yulan He, Yuxuan Xiong, Chenchen Ye, Haiyang Xu