Papers by Xing Lu
LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing (2024.emnlp-main)
Copied to clipboard
Jiangshu Du, Yibo Wang, Wenting Zhao, Zhongfen Deng, Shuaiqi Liu, Renze Lou, Henry Zou, Pranav Narayanan Venkit, Nan Zhang, Mukund Srinath, Haoran Zhang, Vipul Gupta, Yinghui Li, Tao Li, Fei Wang, Qin Liu, Tianlin Liu, Pengzhi Gao, Congying Xia, Chen Xing, Cheng Jiayang, Zhaowei Wang, Ying Su, Raj Shah, Ruohao Guo, Jing Gu, Haoran Li, Kangda Wei, Zihao Wang, Lu Cheng, Surangika Ranathunga, Meng Fang, Jie Fu, Fei Liu, Ruihong Huang, Eduardo Blanco, Yixin Cao, Rui Zhang, Philip Yu, Wenpeng Yin
| Challenge: | a comparative analysis of paper (meta-)reviews by large language models (LLMs) aims to identify and distinguish LLMs from human activities . |
| Approach: | They present a comparative analysis to identify and distinguish LLM activities from human activities. |
| Outcome: | The proposed analysis aims to improve recognition of instances when someone implicitly uses LLMs for reviewing activities. |
Maximizing the Effectiveness of Larger BERT Models for Compression (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for capturing large BERT models as teachers do not fully exploit the potential advantages of larger teachers. |
| Approach: | They propose a method that leverages a pretrained teacher model to guide the training of a lightweight student model to enhance knowledge transfer. |
| Outcome: | The proposed method enhances knowledge transfer by leveraging a pretrained teacher model to guide the training of a lightweight student model. |
Unified Contextual Query Rewriting (2023.acl-industry)
Copied to clipboard
Yingxue Zhou, Jie Hao, Mukund Rungta, Yang Liu, Eunah Cho, Xing Fan, Yanbin Lu, Vishal Vasudevan, Kellen Gillespie, Zeynab Raeesy
| Challenge: | Large-scale conversational AI agents such as Alexa, Siri, and Google Assistant are becoming increasingly popular in real-world applications to assist users in daily life. |
| Approach: | They propose a unified contextual query rewriting model that unifies QR for friction reduction and contextual carryover . they leverage the text-to-text unified framework which uses independent tasks with weighted loss to account for task importance . |
| Outcome: | The proposed model reduces friction and contextual carryover by using multiple auxiliary tasks. |
Towards Knowledge Checking in Retrieval-augmented Generation: A Representation Perspective (2025.naacl-long)
Copied to clipboard
Shenglai Zeng, Jiankun Zhang, Bingheng Li, Yuping Lin, Tianqi Zheng, Dante Everaert, Hanqing Lu, Hui Liu, Hui Liu, Yue Xing, Monica Xiao Cheng, Jiliang Tang
| Challenge: | Existing studies have shown that LLMs struggle to identify the boundaries of their own knowledge and tend to prioritize external information over internal knowledge learned during pre-training. |
| Approach: | They conduct a comprehensive analysis of LLM representation behaviors and demonstrate the significance of using representations in knowledge checking. |
| Outcome: | The proposed classifiers improve performance even when dealing with noisy knowledge databases. |
RoleMRC: A Fine-Grained Composite Benchmark for Role-Playing and Instruction-Following (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing role-playing datasets mostly contribute to controlling role style and knowledge boundaries, but overlook role-following in instruction-follower scenarios. |
| Approach: | They propose a fine-grained role-playing and instruction-following composite benchmark, named RoleMRC, which includes multi-turn dialogues between ideal roles and humans, including free chats or discussions upon given passages . |
| Outcome: | The proposed model improves instruction-following without compromising general role-playing and reasoning capabilities. |
RealBench: A Chinese Multi-image Understanding Benchmark Close to Real-world Scenarios (2025.findings-emnlp)
Copied to clipboard
Fei Zhao, Chengqiang Lu, Yufan Shen, Qimeng Wang, Yicheng Qian, Haoxin Zhang, Yan Gao, null Yiwu, Yao Hu, Zhen Wu, Shangyu Xing, Xinyu Dai
| Challenge: | RealBench is the first Chinese multimodal multi-image dataset . the dataset contains 9393 samples and 69910 images . |
| Approach: | They propose to create a Chinese multimodal multi-image dataset using 21 models . they use closed-source models that support multi-inputs as well as open-source visual and video models a . |
| Outcome: | The first Chinese multimodal multi-image dataset contains 9393 samples and 69910 images. |
Tell Me What You Don’t Know: Enhancing Refusal Capabilities of Role-Playing Agents via Representation Space Analysis and Editing (2025.findings-acl)
Copied to clipboard
Wenhao Liu, Siyu An, Junru Lu, Muling Wu, Tianlong Li, Xiaohua Wang, Changze Lv, Xiaoqing Zheng, Di Yin, Xing Sun, Xuanjing Huang
| Challenge: | Role-playing Agents (RPAs) struggle to recognize and respond to hard queries that conflict with their role-play knowledge. |
| Approach: | They propose a lightweight representation editing approach that conveniently shifts conflicting requests to the rejection region, thereby enhancing the model’s refusal accuracy. |
| Outcome: | The proposed model improves RPAs’ refusal ability of conflicting requests while maintaining their general role-playing capabilities. |
CL-QR: Cross-Lingual Enhanced Query Reformulation for Multi-lingual Conversational AI Agents (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Existing QR systems that reformulate defective user queries are limited in English due to the scarcity of non-English QR labels. |
| Approach: | They propose a query reformulation method which reformulates defective user queries to improve non-English QR performance. |
| Outcome: | The proposed framework improves non-English QR performance by leveraging abundant reformulation resources in English. |
Value Compass Benchmarks: A Comprehensive, Generative and Self-Evolving Platform for LLMs’ Value Evaluation (2025.acl-demo)
Copied to clipboard
Jing Yao, Xiaoyuan Yi, Shitong Duan, Jindong Wang, Yuzhuo Bai, Muhua Huang, Yang Ou, Scarlett Li, Peng Zhang, Tun Lu, Zhicheng Dou, Maosong Sun, James Evans, Xing Xie
| Challenge: | Current evaluation methods for large language models face two key challenges: 1. evaluation validity and 2. Result interpretation reduce the pluralistic and incommensurable values to one-dimensional scores. |
| Approach: | They propose a platform for comprehensive value diagnosis of large language models (LLMs) that provides a generative evaluation paradigm that automatically creates real-world test items co-evolving with ever-advancing LLMs. |
| Outcome: | The proposed platform provides a framework for comprehensive value diagnosis of large language models (LLMs) with fine-grained scores and case studies across 27 value dimensions for 33 leading LLMs, customized comparisons, and visualized analysis of LLM’s alignment with cultural values. |
Eliminating Biased Length Reliance of Direct Preference Optimization via Down-Sampled KL Divergence (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies attributed verbosity to biased labels, but new research shows that DPO can be effective in mitigating verboses. |
| Approach: | They propose to use a method to reduce the amount of verbosity in LLMs by using a downsampling approach. |
| Outcome: | The proposed approach overcomes the problem of verbosity by reducing the length reliance of the proposed algorithm. |
FIPO: Free-form Instruction-oriented Prompt Optimization with Preference Dataset and Modular Fine-tuning Schema (2025.coling-main)
Copied to clipboard
| Challenge: | naive prompts can enhance the task performance of large language models, but they are resource-intensive. |
| Approach: | They propose an automatic prompt optimization method that refines naive prompts according to task outputs from in-box testing models. |
| Outcome: | The proposed method is based on a large-scale dataset and performed fairly across multiple models. |
HiChunk: Evaluating and Enhancing Retrieval Augmented Generation with Hierarchical Chunking (2026.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation benchmarks for document chunking are inadequate due to evidence sparsity . evaluators are unable to evaluate different chunking methods due to the evidence sparing . |
| Approach: | They propose a QA benchmark for document chunking and a hierarchical document structuring framework for it. |
| Outcome: | The proposed framework improves document chunking quality within reasonable time consumption. |
RecMind: Large Language Model Powered Agent For Recommendation (2024.findings-naacl)
Copied to clipboard
Yancheng Wang, Ziyan Jiang, Zheng Chen, Fan Yang, Yingxue Zhou, Eunah Cho, Xing Fan, Yanbin Lu, Xiaojiang Huang, Yingzhen Yang
| Challenge: | Existing recommendations systems are limited in generalizing to new tasks due to model scale and data size constraints. |
| Approach: | They propose an LLM-powered autonomous recommender agent, RecMind, which is capable of leveraging external knowledge to provide zero-shot personalized recommendations. |
| Outcome: | The proposed model outperforms existing zero/few-shot LLM-based recommendation baseline methods in various tasks and achieves comparable performance to a fully trained recommendation model P5. |
Negating Negatives: Alignment with Human Negative Samples via Distributional Dispreference Optimization (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to steer LLMs towards human preference suffer from noisy positive-negative training pairs. |
| Approach: | They propose a distributional preference optimization method which maximizes discrepancy between dispreferred responses and generated non-negative ones. |
| Outcome: | The proposed method achieves comparable generation quality and surpasses the latest strong baselines in producing less harmful and more informative responses with better training stability and faster convergence. |
Weakly Supervised Contrastive Learning for Chest X-Ray Report Generation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Radiology report generation aims at generating descriptive text from radiology images automatically. |
| Approach: | They propose a weakly supervised contrastive loss method that generates descriptive text from radiology images automatically. |
| Outcome: | The proposed method outperforms previous work on correctness and text generation metrics for two public benchmarks. |
Towards Context-Robust LLMs: A Gated Representation Fine-tuning Approach (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) enhanced with external contexts face challenges in handling imperfect evidence. |
| Approach: | They propose a framework that can balance internal knowledge with external contexts . they propose gating mechanisms and low-rank representation adapters to adjust hidden representations based on a lightweight intervention function . |
| Outcome: | The proposed model can effectively balance internal knowledge with external context, similar to human cognitive processes. |
MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation (2025.emnlp-main)
Copied to clipboard
Weihao Xuan, Rui Yang, Heli Qi, Qingcheng Zeng, Yunze Xiao, Aosong Feng, Dairui Liu, Yun Xing, Junjue Wang, Fan Gao, Jinghui Lu, Yuang Jiang, Huitao Li, Xin Li, Kunyu Yu, Ruihai Dong, Shangding Gu, Yuekang Li, Xiaofei Xie, Felix Juefei-Xu, Foutse Khomh, Osamu Yoshie, Qingyu Chen, Douglas Teodoro, Nan Liu, Randy Goebel, Lei Ma, Edison Marrese-Taylor, Shijian Lu, Yusuke Iwasawa, Yutaka Matsuo, Irene Li
| Challenge: | Existing large language model evaluation benchmarks focus on English, while current multilingual tasks lack parallel questions that specifically assess cross-lingual reasoning abilities. |
| Approach: | They propose a comprehensive benchmark covering 29 languages, built on an English benchmark. |
| Outcome: | The MMLU-ProX is a comprehensive benchmark covering 29 languages, built on an English benchmark. |
Mitigating the Privacy Issues in Retrieval-Augmented Generation (RAG) via Pure Synthetic Data (2025.emnlp-main)
Copied to clipboard
Shenglai Zeng, Jiankun Zhang, Pengfei He, Jie Ren, Tianqi Zheng, Hanqing Lu, Han Xu, Hui Liu, Yue Xing, Jiliang Tang
| Challenge: | Existing literature suggests that RAG systems may face privacy issues when the retrieval process involves private data. |
| Approach: | They propose a two-stage synthetic data generation paradigm that uses attributes to preserve contextual information from the original data. |
| Outcome: | The proposed approach preserves key contextual information from the original data while reducing privacy risks. |
Thermometer of Thoughts: Enhancing LLM’s Exploration via Attention Temperature Modulation (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in the reasoning capabilities of large language models have enabled them to tackle complex tasks such as mathematics reasoning. |
| Approach: | They propose a method that modulates attention temperature dynamically to shift LLM's internal focus during reasoning, enabling a dynamic shift between exploratory and focused modes. |
| Outcome: | The proposed method improves Pass@10 by 6.78–14.20% and aggregation accuracy by 9.74% across 7 reasoning benchmarks. |
AIGT: AI Generative Table Based on Prompt (2025.coling-main)
Copied to clipboard
| Challenge: | Tabular data is an essential resource for many fields, but current methods do not fully utilize the rich information available in tables. |
| Approach: | They propose a method that utilizes metadata information to generate tabular data . they propose long-token partitioning algorithms that enable AIGT to model tables of any scale . |
| Outcome: | The proposed approach achieves state-of-the-art on 14 out of 20 public datasets and two real industry datasets within the Alipay risk control system. |
Aligning Large Language Models for Controllable Recommendations (2024.acl-long)
Copied to clipboard
| Challenge: | Existing literature focuses on integrating domain-specific knowledge into LLMs to enhance accuracy using a fixed task template. |
| Approach: | They propose a collection of supervised learning tasks augmented with labels derived from a conventional recommender model to improve LLMs’ proficiency in adhering to recommendation-specific instructions. |
| Outcome: | The proposed approach significantly improves the capability of LLMs to respond to instructions within recommender systems, reducing formatting errors while maintaining a high level of accuracy. |
Graph Meets LLM: A Novel Approach to Collaborative Filtering for Robust Conversational Understanding (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Defective queries impact the robustness of conversational AI systems such as Alexa, Siri or Google Assistant. |
| Approach: | They propose a Personalized Query Rewriting system that takes into account individual preferences or unique error patterns identified from a user's historical interactions with the conversational AI. |
| Outcome: | The proposed approach has been proven on a large-scale real-world dataset and online A/B experiments. |
Phun-Bench: Evaluating LLMs on Phonological Understanding in Chinese (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks on LLMs’ phonological abilities are either solvable through rote memorization or intertwined with other abilities, making them inadequate to measure LLM’s genuine ability in *phonological understanding*. |
| Approach: | They propose to use a Chinese benchmark to evaluate LLMs' phonological understanding to test their ability to recall correct pronunciations. |
| Outcome: | The proposed benchmarks show that LLMs excel at recalling correct pronunciations, but struggle to leverage phonological knowledge in the flexible and intuitive way that human speakers do. |
Improving Contextual Query Rewrite for Conversational AI Agents through User-preference Feedback Learning (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Contextual query rewriting (CQR) is a crucial component in Conversational AI agents, leveraging contextual information from previous user-agent conversations to improve comprehension of current user intent. |
| Approach: | They propose a framework to enhance the CQR model's capability in generating user preference-aligned rewrites. |
| Outcome: | The proposed framework improves the CQR model's ability to generate user preference-aligned rewrites. |
DongbaMIE: A Multimodal Information Extraction Dataset for Evaluating Semantic Understanding of Dongba Pictograms (2025.findings-emnlp)
Copied to clipboard
Xiaojun Bi, Shuo Li, Junyao Xing, Ziyue Wang, Fuwen Luo, Weizheng Qiao, Lu Han, Ziwei Sun, Peng Li, Yang Liu
| Challenge: | Dongba pictographic is the only pictograph script still in use in the world. |
| Approach: | DongbaMIE is the first dataset focusing on multimodal information extraction of Dongbe pictographs. |
| Outcome: | The dataset contains 23,530 sentence-level and 2,539 paragraph-level high-quality text-image pairs. |
VisLingInstruct: Elevating Zero-Shot Learning in Multi-Modal Language Models with Autonomous Instruction Optimization (2024.naacl-long)
Copied to clipboard
Dongsheng Zhu, Daniel Tang, Weidong Han, Jinghui Lu, Yukun Zhao, Guoliang Xing, Junfeng Wang, Dawei Yin
| Challenge: | Current MMLMs show impressive zero-shot abilities in multi-modal tasks, but their performance depends heavily on the quality of instructions. |
| Approach: | They propose a novel approach to advancing multi-modal language models in zero-shot learning by evaluating and optimizing instructional texts through In-Context Learning. |
| Outcome: | The proposed approach improves zero-shot performance in multi-modal tasks by evaluating and optimizing instructional texts. |