Papers by Ye Ren

Grafting Pre-trained Models for Multimodal Headline Generation (2022.emnlp-industry)

Challenge:	Biology-Instructions is the first large-scale instruction-tuning dataset for multi-omics biological sequences.
Approach:	They propose a large-scale instruction-tuning dataset for multi-omics biological sequences . they propose 'chatMultiOmics' to overcome limitations of current LLMs on multi-ome tasks .
Outcome:	The proposed dataset bridges LLMs and complex biological sequence-related tasks while maintaining conversational fluency.

Copied to clipboard

Lingfeng Qiao, Chen Wu, Ye Liu, Haoyuan Peng, Di Yin, Bo Ren

GeAR: Graph-enhanced Agent for Retrieval-augmented Generation (2025.findings-acl)

Challenge:	Existing approaches to generate video headlines with pre-trained language models are labor intensive and impractical.
Approach:	They propose to graft the encoder from the pre-trained video-language model on the generative pre-trainer model and propose a consensus fusion mechanism for the integration of different components.
Outcome:	The proposed model achieves strong results on a brand-new dataset collected from real-world applications.

Copied to clipboard

Zhili Shen, Chenxin Diao, Pavlos Vougiouklis, Pascual Merita, Shriram Piramanayagam, Enting Chen, Damien Graux, Andre Melo, Ruofei Lai, Zeren Jiang, Zhongyang Li, Ye Qi, Yang Ren, Dandan Tu, Jeff Z. Pan

Generative Psycho-Lexical Approach for Constructing Value Systems in Large Language Models (2025.acl-long)

Challenge:	Retrieval-augmented Generation (RAG) relies on effective retrieval capabilities, yet traditional sparse and dense retrievers struggle with multi-hop retrieval scenarios.
Approach:	They propose a graph expansion mechanism that augments any conventional base retriever and an agent framework that incorporates the resulting graph-based retrieval into a multi-step retrieval framework.
Outcome:	The proposed system achieves state-of-the-art results on three multi-hop question answering datasets while consuming fewer tokens and requiring fewer iterations than existing multi-step retrieval systems.

Copied to clipboard

Haoran Ye, TianZe Zhang, Yuhang Xie, Liyuan Zhang, Yuanyi Ren, Xin Zhang, Guojie Song

KELE: Residual Knowledge Erasure for Enhanced Multi-hop Reasoning in Knowledge Editing (2025.findings-emnlp)

Challenge:	Large Language Models (LLMs) have raised concerns regarding their intrinsic values.
Approach:	They propose a psychologically grounded five-factor value system for Large Language Models that integrates psychological principles with cutting-edge AI priorities.
Outcome:	The proposed value system meets standard psychological criteria, improves LLM safety prediction, and enhances Llm alignment, when compared to the canonical Schwartz’s values.

Copied to clipboard

Mengqi Zhang, Bowen Fang, Qiang Liu, Xiaotian Ye, Shu Wu, Pengjie Ren, Zhumin Chen, Liang Wang

VistaWise: Building Cost-Effective Agent with Cross-Modal Knowledge Graph for Minecraft (2025.emnlp-main)

Challenge:	Existing knowledge editing techniques show limitations when applied to multi-hop reasoning . residual single-hop knowledge causes edited models to revert to original answers .
Approach:	They propose a knowledge editing method that incorporates a Knowledge Erasure mechanism for Large language model Editing (KELE) they propose an erasure function for residual knowledge and an injection function for new knowledge .
Outcome:	The proposed method significantly improves multi-hop reasoning capability of edited models.

Copied to clipboard

Honghao Fu, Junlong Ren, Qi Chai, Deheng Ye, Yujun Cai, Hao Wang

LLM-driven Instruction Following: Progresses and Concerns (2023.emnlp-tutorial)

Challenge:	Large language models (LLMs) have shown significant promise in embodied decision-making tasks within virtual open-world environments, but lack domain-specific knowledge.
Approach:	They propose a cost-effective agent framework that integrates cross-modal domain knowledge and finetunes a dedicated object detection model for visual analysis.
Outcome:	The proposed framework reduces the requirement for domain-specific training data from millions of samples to a few hundred.

Copied to clipboard

Wenpeng Yin, Qinyuan Ye, Pengfei Liu, Xiang Ren, Hinrich Schütze

Challenge:	a tutorial on task instruction is aimed at researchers and practitioners interested in NLP generalization . labeled examples are unlikely to be available in large numbers or do not exist .
Approach:	This tutorial will examine the progress of natural language processing (NLP) using labeled examples. authors propose that task instructions act as a novel resource for supervision.
Outcome:	This tutorial aims to answer questions about instruction-driven NLP . it focuses on the use of task instructions in a low-shot scenario .

LEAN-LIFE: A Label-Efficient Annotation Framework Towards Learning from Explanation (2020.acl-demos)

Copied to clipboard

Dong-Ho Lee, Rahul Khanna, Bill Yuchen Lin, Seyeon Lee, Qinyuan Ye, Elizabeth Boschee, Leonardo Neves, Xiang Ren

1+1>2: A Synergistic Sparse and Low-Rank Compression Method for Large Language Models (2025.findings-emnlp)

Challenge:	Existing frameworks for sequence labeling and classification require massive human effort and labeling data is limited.
Approach:	They propose a web-based, Label-Efficient AnnotatioN framework that allows an annotator to provide the needed labels for a task and can capture explanations for each labeling decision.
Outcome:	The proposed framework surpasses baseline F1 scores by 5-10 percentage points while using 2X times fewer labeled instances.

Copied to clipboard

Zeliang Zong, Kai Zhang, Zheyang Li, Wenming Tan, Ye Ren, Yiyan Zhai, Jilin Hu

Unlocking Multilingual Reasoning Capability of LLMs and LVLMs through Representation Engineering (2026.acl-long)

Challenge:	Low-rank approximation compresses the model by retaining its essential structure with minimal information loss.
Approach:	They propose a method that leverages the strengths of pruning and low-rank approximation for LLMs.
Outcome:	The proposed methods surpass the existing methods on LLaMA and Qwen2.5 models.

Copied to clipboard

Qiming Li, Xiaocheng Feng, Yixuan Ma, Ruihan Chen, Zihe Tong, Zekai Ye, Xiachong Feng, Libo Qin, Haoyu Ren, Kun Chen, Yunfei Lu, Dandan Tu, Bing Qin

Automatic and Reliable Evaluation for Academic Caption-to-Figure Generation with LMMs (2026.acl-long)

Challenge:	Existing approaches to enhance multilingual reasoning capabilities rely on costly multilingual training or employ prompting with external translation tools.
Approach:	They propose a training-free inference-time method to enhance multilingual reasoning capabilities via Representation Engineering without additional training data or tools.
Outcome:	The proposed method outperforms existing methods on four reasoning benchmarks in English and Thai and Swahili.

Copied to clipboard

Guanghui Ye, Huan Zhao, Qin Zhu, Fengnan Li, Jiaqi Li, Yixian Shen, Zhonghao Ren, Zhihua Jiang

Social Welfare Function Leaderboard: On the Emergence of LLM Agents as the Welfare Dictator (2026.findings-acl)

Challenge:	Existing datasets for evaluating text-to-image generation focus mostly on real-life images, which poses challenges for assessing academic figure generation given real scientific captions.
Approach:	They propose a dataset that first provides a Holistic Evaluation for Academic caption-to-Figure Generation (HE4AFG) they collect real figure captions from 8 scientific domains and generate 3,900 evaluation samples .
Outcome:	The proposed model provides high-quality human ratings in terms of three aspects—scientific aesthetic (SA), topic relevance (TR), and attribute correctness (AC).

Copied to clipboard

Zhengliang Shi, Ruotian Ma, Jen-tse Huang, Xinbei Ma, Xingyu Chen, Mengru Wang, Qu Yang, Yue Wang, Fanghua Ye, Ziyang Chen, Shanyi Wang, Cixing LI, Wenxuan Wang, Zhaopeng Tu, Xiaolong Li, Zhaochun Ren, Liefeng Bo

Estimating Large Language Model Capabilities without Labeled Test Data (2023.findings-emnlp)

Challenge:	Large language models (LLMs) are increasingly entrusted with high-stakes decisions that affect human welfare.
Approach:	They evaluate 20 state-of-the-art Large language models (LLMs) and 20 LLM dictators to create a social welfare function benchmark.
Outcome:	The proposed model creates dilemma between maximizing collective efficiency and ensuring distributive fairness.

Copied to clipboard

Harvey Fu, Qinyuan Ye, Albert Xu, Xiang Ren, Robin Jia

FiD-ICL: A Fusion-in-Decoder Approach for Efficient In-Context Learning (2023.acl-long)

Challenge:	Large Language Models have shown impressive ability to perform in-context learning from only a few examples, but their accuracy varies widely from task to task.
Approach:	They propose a method that trains a meta-model using LLM confidence scores as features to perform ICL accuracy estimation.
Outcome:	The proposed method improves over baselines across 7 out of 12 settings and achieves the same accuracy as evaluating on 40 sampled examples per task.

Copied to clipboard

Qinyuan Ye, Iz Beltagy, Matthew Peters, Xiang Ren, Hannaneh Hajishirzi

Sparse Distillation: Speeding Up Text Classification by Using Bigger Student Models (2022.naacl-main)

Challenge:	Large pre-trained models are capable of few-shot in-context learning (ICL) however, concatenated demonstrations are often excessively long and require additional computation.
Approach:	They propose to apply fusion-in-decoder (FiD) models to perform few-shot in-context learning (ICL) they propose to use concatenation-based, early-fusion, intermediate- and late-fusion methods to improve efficiency .
Outcome:	The proposed methods outperform concatenation-based models on 11 held-out tasks.

Copied to clipboard

Qinyuan Ye, Madian Khabsa, Mike Lewis, Sinong Wang, Xiang Ren, Aaron Jaech

Teaching Machine Comprehension with Compositional Explanations (2020.findings-emnlp)

Challenge:	Existing methods to reduce inference cost by distilling transformer models into lightweight student models are limited for high-volume use cases.
Approach:	They propose to distill state-of-the-art transformer models into lightweight student models to reduce computation cost at inference time.
Outcome:	The proposed pipeline achieves up to 600x speed-up on GPUs and CPUs on six single-sentence text classification tasks and in domain generalization settings.

Copied to clipboard

Qinyuan Ye, Xiao Huang, Elizabeth Boschee, Xiang Ren

Explainability & Interpretability in NLP

Challenge:	Recent advances in machine reading comprehension rely heavily on large-scale annotated corpora, which are timeconsuming and costly to collect.
Approach:	They propose to use semi-structured explanations to “teach” machines reading comprehension using a small number of semi-structural explanations that explicitly inform machines why answer spans are correct.
Outcome:	The proposed method achieves 70.14% F1 score with supervision from 26 explanations on the SQuAD dataset, comparable to plain supervised learning using 1,100 labeled instances yielding a 12x speed up.

CodeTool: Enhancing Programmatic Tool Invocation of LLMs via Process Supervision (2025.acl-long)

Copied to clipboard

YifeiLu YifeiLu, Fanghua Ye, Jian Li, Qiang Gao, Cheng Liu, Haibo Luo, Nan Du, Xiaolong Li, Feiliang Ren

Programming Languages in NLP

Challenge:	Existing approaches to tool invocation are often unnecessarily long and require lengthy reasoning paths.
Approach:	They propose a framework for stepwise code generation that improves LLM tool invocation . they incorporate two distinct process rewards: the On-the-spot and the Latent Reward .
Outcome:	The proposed framework improves LLM tool invocation by leveraging the concise nature of code.

Fundamental Capabilities of Large Language Models and their Applications in Domain Scenarios: A Survey (2024.acl-long)

Copied to clipboard

Jiawei Li, Yizhe Yang, Yu Bai, Xiaofeng Zhou, Yinghao Li, Huashan Sun, Yuhang Liu, Xingpeng Si, Yuhao Ye, Yixiao Wu, 林一冠林一冠, Bin Xu, Ren Bowen, Chong Feng, Yang Gao, Heyan Huang

Learning to Generate Task-Specific Adapters from Task Description (2021.acl-short)

Challenge:	Existing benchmark-based evaluations cannot accurately reflect the performance of real-world applications.
Approach:	They propose a reliable strategy for domains to choose more robust LLMs for real-world applications.
Outcome:	The proposed strategy addresses the challenges faced by domains to choose more robust LLMs for real-world applications.

Copied to clipboard

Qinyuan Ye, Xiang Ren

Spectral Characterization and Mitigation of Sequential Knowledge Editing Collapse (2026.acl-long)

Challenge:	Pre-trained text-to-text transformers have achieved impressive performance across a range of NLP tasks, such as question answering and commonsense reasoning.
Approach:	They propose a framework that improves text-to-text transformer’s generalization ability to unseen tasks by training a hypernetwork to generate task-specific adapters from task descriptions.
Outcome:	Experiments on ZEST and a synthetic SQuAD dataset show that Hypter improves upon fine-tuning baselines.

Copied to clipboard

Chi Zhang, Mengqi Zhang, Xiaotian Ye, Runxi Cheng, Zisheng Zhou, Ying Zhou, Pengjie Ren, Zhumin Chen

Challenge:	Existing approaches to reducing the effects of knowledge editing are insufficiently understood.
Approach:	They propose a plug-and-play framework that preserves the dominant subspace of the original weights and analyzes parameter updates in the spectral basis of the weights.
Outcome:	The proposed framework improves editing efficacy while preserving general abilities under long-horizon sequential editing, including extreme settings with up to 20,000 edits.

CausalMACE: Causality Empowered Multi-Agents in Minecraft Cooperative Tasks (2025.findings-emnlp)

Copied to clipboard

Qi Chai, Zhang Zheng, Junlong Ren, Deheng Ye, Zichuan Lin, Hao Wang

Challenge:	Existing work on multi-agent collaborative tasks in Minecraft is limited due to inefficiency and limited fault tolerance.
Approach:	They propose a framework that incorporates causality to manage dependencies among subtasks.
Outcome:	The proposed framework achieves state-of-the-art performance in multi-agent cooperative tasks of Minecraft.

Looking Beyond Label Noise: Shifted Label Distribution Matters in Distantly Supervised Relation Extraction (D19-1)

Copied to clipboard

Qinyuan Ye, Liyuan Liu, Maosen Zhang, Xiang Ren

Information Extraction & Text Mining

Challenge:	Existing studies on DS-based relation extraction (RE) methods focus on handling label noise, but other factors may have been overlooked.
Approach:	They propose a method to automatically adjust DS-RE models to a shifted label distribution problem . they find this problem exists in real-world DS datasets and can be overcome .
Outcome:	The proposed method achieves consistent performance gains on DS-trained models with an up to 23% relative F1 improvement, which verifies their assumptions.

CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP (2021.emnlp-main)

Copied to clipboard

Qinyuan Ye, Bill Yuchen Lin, Xiang Ren

Grounded Multimodal Procedural Entity Recognition for Procedural Documents: A New Dataset and Baseline (2024.lrec-main)

Challenge:	We study whether and how cross-task generalization ability can be acquired . we use CrossFit to standardize seen/unseen task partitions and evaluation protocols .
Approach:	They propose a problem setup for studying cross-task generalization ability which standardizes seen/unseen task partitions and data access during different learning stages.
Outcome:	The proposed model can be used to build few-shot learners across diverse tasks.

Copied to clipboard

Haopeng Ren, Yushi Zeng, Yi Cai, Zhenqi Ye, Li Yuan, Pinli Zhu

Information Extraction & Text Mining

Challenge:	Existing methods to extract procedural knowledge from documents focus on text-only settings, which is insufficient for entity disambiguation.
Approach:	They propose a model to detect the entity and the corresponding bounding box groundings in images.
Outcome:	The proposed model detects the entity and the corresponding bounding box groundings in image (i.e., visual entities) it is based on a dataset of a WikiHow 1 and EHow 2 document and the results are compared with existing models.

Knowledge Graph Enhanced Large Language Model Editing (2024.emnlp-main)

Copied to clipboard

Mengqi Zhang, Xiaotian Ye, Qiang Liu, Pengjie Ren, Shu Wu, Zhumin Chen

UIPE: Enhancing LLM Unlearning by Removing Knowledge Related to Forgetting Targets (2025.findings-emnlp)

Challenge:	Existing methods for editing large language models struggle to track and incorporate changes in knowledge associated with edits, which limits the generalization ability of post-edit LLMs in processing edited knowledge.
Approach:	They propose a model editing method that leverages knowledge graphs to enhance LLM editing by capturing changes in associated knowledge by constructing an external graph.
Outcome:	The proposed method improves the generalization ability of LLMs in processing edited knowledge.

Copied to clipboard

Wenyu Wang, Mengqi Zhang, Xiaotian Ye, Zhaochun Ren, Pengjie Ren, Zhumin Chen

Specializing Word Vectors by Spectral Decomposition on Heterogeneously Twisted Graphs (2020.coling-main)

Challenge:	Existing methods to unlearning large language models focus on forgetting target data while overlooking the impact of logically related knowledge on the effectiveness of unlearning.
Approach:	They propose a method that removes knowledge highly correlated with the forgetting targets and a technique that remove logically related knowledge from the model.
Outcome:	The proposed method significantly improves the performance of the proposed method on the TOFU and WMDP benchmarks.

Copied to clipboard

Yuanhang Ren, Ye Du

FluentSpeech: Stutter-Oriented Automatic Speech Editing with Context-Aware Diffusion Models (2023.findings-acl)

Challenge:	Word vectors have a tendency to conflate semantic similarity with semantic relatedness . a new method is proposed to retrofit word vectors with lexical constraints .
Approach:	They propose a method that heterogeneously retrofits a similarity matrix with lexical constraints.
Outcome:	The proposed method has a competitive performance compared with the state-of-the-art methods.

Copied to clipboard

Ziyue Jiang, Qian Yang, Jialong Zuo, Zhenhui Ye, Rongjie Huang, Yi Ren, Zhou Zhao

On the Influence of Masking Policies in Intermediate Pre-training (2021.emnlp-main)

Challenge:	Existing methods for speech editing still suffer from over-smoothing problem and lack of robustness due to stutter.
Approach:	They propose a stutter-oriented automatic speech editing model that incorporates sutter information into the hidden sequence.
Outcome:	The proposed model achieves state-of-the-art performance on a speech recording dataset . it can improve fluency of stuttering speech in terms of objective and subjective metrics.

Copied to clipboard

Qinyuan Ye, Belinda Z. Li, Sinong Wang, Benjamin Bolte, Hao Ma, Wen-tau Yih, Xiang Ren, Madian Khabsa

CogDual: Enhancing Dual Cognition of LLMs via Reinforcement Learning with Implicit Rule-Based Rewards (2025.emnlp-main)

Challenge:	Existing studies show that inserting an intermediate pre-training stage improves performance of masked language models.
Approach:	They propose methods to automate the discovery of optimal masking policies via direct supervision or meta-learning.
Outcome:	The proposed method outperforms the heuristic of masking named entities on TriviaQA and can be generalizable beyond that task.

Copied to clipboard

Cheng Liu, Yifei Lu, Fanghua Ye, Jian Li, Xingyu Chen, Feiliang Ren, Zhaopeng Tu, Xiaolong Li

Challenge:	Existing approaches to role-playing language models rely on prompt engineering or supervised fine-tuning to emulate character behaviors but neglect the underlying cognitive mechanisms driving these behaviors.
Approach:	They propose a novel RPLA adopting a cognize-then-respond reasoning paradigm that leverages dual cognition for more contextually grounded and psychologically coherent responses.
Outcome:	The proposed RPLA outperforms baselines and generalizes effectively across diverse role-playing tasks.

ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language Models (2024.acl-long)

Copied to clipboard

Yuanyi Ren, Haoran Ye, Hanjun Fang, Xin Zhang, Guojie Song

How Predictable Are Large Language Model Capabilities? A Case Study on BIG-bench (2023.findings-emnlp)

Challenge:	Large Language Models (LLMs) are transforming diverse fields and gaining increasing influence as human proxies.
Approach:	They propose a psychometric evaluation pipeline grounded in realistic human-AI interactions to probe value orientations and novel tasks for evaluating value understanding in an open-ended value space.
Outcome:	The proposed evaluation pipeline is grounded in realistic human-AI interactions and performs tasks that approximate expert conclusions in value-related extraction and generation tasks.

Copied to clipboard

Qinyuan Ye, Harvey Fu, Xiang Ren, Robin Jia

Autonomous Workflow for Multimodal Fine-Grained Training Assistants Towards Mixed Reality (2024.findings-acl)

Challenge:	a recent study shows that large language models can be used to predict performance on new configurations.
Approach:	They investigate the predictability of large language model capabilities by using BIG-bench . they find a subset of BIG-Bench tasks as informative as BIG-bnch Hard .
Outcome:	The proposed model achieves an R2 score greater than 95% on BIG-bench . the model is 3 smaller than BIG-Bench Hard, and the model performs better on the full set.

Copied to clipboard

Jiahuan Pei, Irene Viola, Haochen Huang, Junxiao Wang, Moonisa Ahsan, Fanghua Ye, Jiang Yiming, Yao Sai, Di Wang, Zhumin Chen, Pengjie Ren, Pablo Cesar

CLAPSpeech: Learning Prosody from Text Context with Contrastive Language-Audio Pre-Training (2023.acl-long)

Challenge:	a fine-grained, comprehensive understanding of multimodal environments remains under-explored.
Approach:	They propose an automated workflow for integrating AI agents into extended reality (XR) they propose a cerebral language agent that integrates LLM with memory, planning, and interaction with XR tools and a vision-language agent .
Outcome:	The proposed workflow integrates AI agents seamlessly into extended reality (XR) applications for fine-grained training.

Copied to clipboard

Zhenhui Ye, Rongjie Huang, Yi Ren, Ziyue Jiang, Jinglin Liu, Jinzheng He, Xiang Yin, Zhou Zhao

AV-TranSpeech: Audio-Visual Robust Speech-to-Speech Translation (2023.acl-long)

Challenge:	Existing methods for expressive text-to-speech only implicitly learn prosody with masked token reconstruction tasks.
Approach:	They propose a cross-modal contrastive pre-training framework that learns from prosody variance of the same text token under different contexts.
Outcome:	The proposed framework can learn from prosody variance of a text token under different contexts.

Copied to clipboard

Rongjie Huang, Huadai Liu, Xize Cheng, Yi Ren, Linjun Li, Zhenhui Ye, Jinzheng He, Lichao Zhang, Jinglin Liu, Xiang Yin, Zhou Zhao

Beyond Dynamic Quantization: An Efficient Static Hierarchical Mix-precision Framework for Near-Lossless LLM Compression (2025.emnlp-industry)

Challenge:	Existing models for speech-to-speech translation suffer from distinct degradation in noisy environments and fail to translate visual speech.
Approach:	They propose a text-based audio-visual speech-to-speech translation model that integrates visual information with audio-only data to improve system robustness.
Outcome:	The proposed model outperforms models trained on audio-only corpus in two languages . it also improves with low-resource audio-visual data, compared with baselines .

Copied to clipboard

Yi Zhang, Kai Zhang, Zheyang Li, Wenming Tan, Ye Ren, Jilin Hu

Eliciting and Understanding Cross-task Skills with Task-level Mixture-of-Experts (2022.findings-emnlp)

Challenge:	Existing methods for dynamic quantization are hardware-unfriendly and often lead to large quantization errors in static scenarios.
Approach:	They propose a Static Hierarchical Mix-precision Quantization method which quantifies both inter-layer and intra-layer sensitivity through unified derivations involving Hessian.
Outcome:	The proposed method achieves 75.58% on zero-shot reasoning tasks while yielding average speedup of 2.86.

Copied to clipboard

Qinyuan Ye, Juan Zha, Xiang Ren