Papers by Xiaoqing Zheng

Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitution (2021.emnlp-main)

Challenge:	Large language models (LLMs) demonstrate remarkable performance across diverse tasks, yet their effectiveness often depends on costly commercial APIs or cloud services.
Approach:	They propose a dual-mode compatible approach that fine-tunes models through shortest-response preference optimization and a confidence-aware rejection mechanism.
Outcome:	The proposed approach reduces redundant outputs and response times while reducing computational costs by over 50% and cascade latency by over 80%.

Copied to clipboard

Zongyi Li, Jianhan Xu, Jiehang Zeng, Linyang Li, Xiaoqing Zheng, Qi Zhang, Kai-Wei Chang, Cho-Jui Hsieh

Information Retrieval

Challenge:	Existing methods to defend against adversarial word-substitution attacks have not been evaluated or compared in a systematic manner.
Approach:	They propose to compare different defense methods under representative adversarial attacks . they propose a method that improves the robustness of neural text classifiers against such attacks a .
Outcome:	The proposed method improves robustness of neural text classifiers against such attacks by a significant margin.

TripTailor: A Real-World Benchmark for Personalized Travel Planning (2025.findings-acl)

Copied to clipboard

Kaimin Wang, Yuanzhe Shen, Changze Lv, Xiaoqing Zheng, Xuanjing Huang

Challenge:	Existing evaluation metrics for travel planning rely on unrealistic simulated data . fewer than 10% of the itineraries generated by the latest state-of-the-art LLMs achieve human-level performance.
Approach:	They propose a benchmark for personalized travel planning in real-world scenarios . they identify several critical challenges in travel planning including feasibility and rationality .
Outcome:	The proposed benchmarks show that fewer than 10% of the itineraries generated by the latest state-of-the-art LLMs achieve human-level performance.

Enhancing Model Privacy in Federated Learning with Random Masking and Quantization (2025.findings-emnlp)

Copied to clipboard

Zhibo Xu, Zhu JianHao, Jingwen Xu, Changze Lv, Zhenghua Wang, Zisu Huang, Xiaohua Wang, Muling Wu, Qi Qian, Xiaoqing Zheng, Xuanjing Huang

Aligning Large Language Models with Human Preferences through Representation Engineering (2024.acl-long)

Challenge:	federated learning approaches are limited by the complexity of large language models and the need for specialized expertise to protect intellectual property.
Approach:	They propose a federated learning approach that leverages random masking to obscure a subnetwork of model parameters and applies quantization to the remaining parameters.
Outcome:	The proposed approach maintains strong model performance in federated learning settings and achieves enhanced protection of model parameters compared to baseline methods.

Copied to clipboard

Wenhao Liu, Xiaohua Wang, Muling Wu, Tianlong Li, Changze Lv, Zixuan Ling, Zhu JianHao, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang

Improving Grammatical Error Correction Models with Purpose-Built Adversarial Examples (2020.emnlp-main)

Challenge:	Existing methods for achieving this alignment involve employing reinforcement learning from human feedback (RLHF) Existing approaches involve using RLHF to fine-tune LLMs based on human labels . however, RLRF is susceptible to instability during fine- tuning and presents challenges in implementation.
Approach:	They propose to use reinforcement learning from human feedback to fine-tune large language models with human preferences to achieve precise control of model behavior.
Outcome:	Experiments show that RAHF can be used to capture and manipulate representations to align with a broad spectrum of human preferences or values rather than being confined to a single concept or function.

Copied to clipboard

Lihao Wang, Xiaoqing Zheng

Improving Empathetic Dialogue Generation by Dynamically Infusing Commonsense Knowledge (2023.findings-acl)

Challenge:	Existing methods for grammatical error correction are data-hungry and it is hard to train a seq2seq model with good performance without suf-Clean.
Approach:	They propose a method inspired by adversarial training to generate more meaningful and valuable training examples by continually identifying weak spots of a model and to enhance the model by gradually adding adversarials to the training set.
Outcome:	The proposed method improves generalization and robustness of GEC models by adding adversarial examples to the training set.

Copied to clipboard

Hua Cai, Xuli Shen, Qing Xu, Weilin Shen, Xiaomei Wang, Weifeng Ge, Xiaoqing Zheng, Xiangyang Xue

Dialogue Systems & Conversational Agents

Challenge:	Existing work on generating empathetic responses by utilizing the speaker's emotion has not been successful.
Approach:	They propose an approach which incorporates an adaptive module for commonsense knowledge selection to ensure consistency between the generated empathetic responses and the speaker’s situation.
Outcome:	The proposed approach outperforms baseline models in both automatic and human evaluations, exhibiting the generation of more coherent and empathetic responses.

Parameter Efficient Multi-task Fine-tuning by Learning to Transfer Token-wise Prompts (2023.findings-emnlp)

Copied to clipboard

Muling Wu, Wenhao Liu, Jianhan Xu, Changze Lv, Zixuan Ling, Tianlong Li, Longtao Huang, Xiaoqing Zheng, Xuanjing Huang

Responsible & Trustworthy NLP

Challenge:	Prompt tuning has been proven to be successful on various tasks by incorporating a small number of trainable parameters while freezing large pre-trained language models.
Approach:	They propose a token-wise prompt tuning method that uses a bank of finer-grained soft prompt tokens to generate an instance-dependent prompt.
Outcome:	The proposed method performs far better than full parameter fine-tuned models and achieves state-of-the-art by tuning only 0.035% parameters on 14 datasets.

Tell Me What You Don’t Know: Enhancing Refusal Capabilities of Role-Playing Agents via Representation Space Analysis and Editing (2025.findings-acl)

Copied to clipboard

Wenhao Liu, Siyu An, Junru Lu, Muling Wu, Tianlong Li, Xiaohua Wang, Changze Lv, Xiaoqing Zheng, Di Yin, Xing Sun, Xuanjing Huang

Representation Learning

Challenge:	Role-playing Agents (RPAs) struggle to recognize and respond to hard queries that conflict with their role-play knowledge.
Approach:	They propose a lightweight representation editing approach that conveniently shifts conflicting requests to the rejection region, thereby enhancing the model’s refusal accuracy.
Outcome:	The proposed model improves RPAs’ refusal ability of conflicting requests while maintaining their general role-playing capabilities.

Searching for Best Practices in Retrieval-Augmented Generation (2024.emnlp-main)

Copied to clipboard

Xiaohua Wang, Zhenghua Wang, Xuan Gao, Feiran Zhang, Yixin Wu, Zhibo Xu, Tianyuan Shi, Zhengyuan Wang, Shizheng Li, Qi Qian, Ruicheng Yin, Changze Lv, Xiaoqing Zheng, Xuanjing Huang

Information Retrieval

Challenge:	Retrieval-augmented generation (RAG) techniques have proven to be effective in integrating up-to-date information, mitigating hallucinations, and enhancing response quality, especially in specialized domains.
Approach:	They propose several strategies for deploying RAG that balance performance and efficiency.
Outcome:	The proposed approaches can significantly enhance question-answering capabilities and accelerate the generation of multimodal content using a “retrieval as generation” strategy.

Evaluating and Enhancing the Robustness of Neural Network-based Dependency Parsing Models with Adversarial Examples (2020.acl-main)

Copied to clipboard

Xiaoqing Zheng, Jiehang Zeng, Yi Zhou, Cho-Jui Hsieh, Minhao Cheng, Xuanjing Huang

Cross-Lingual Dependency Parsing by POS-Guided Word Reordering (2020.findings-emnlp)

Challenge:	Previously studies focused on semantic tasks such as sentiment analysis, question answering and reading comprehension.
Approach:	They propose two approaches to study where and how adversarial examples exist in dependency parsing . they use a state-of-the-art parser to find adversarials in existing texts .
Outcome:	The proposed approaches show that adversarial examples exist in dependency parsing . they show that up to 77% of input examples admit adversarials .

Copied to clipboard

Lu Liu, Yi Zhou, Jianhan Xu, Xiaoqing Zheng, Kai-Wei Chang, Xuanjing Huang

Syntactic Text Processing

Challenge:	Existing approaches to cross-lingual dependency parsing rely on large corpus size and cost.
Approach:	They propose a cross-lingual dependency parsing approach based on word reordering . they propose to train a model that transfers knowledge learned in one or multiple languages to target languages .
Outcome:	The proposed approach outperforms the baseline approach in Hindi and Latin by 15.3% and 6.7%.

Exploration and Exploitation: Two Ways to Improve Chinese Spelling Correction Models (2021.acl-short)

Copied to clipboard

Chong Li, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang

On the Transferability of Adversarial Attacks against Neural Text Classifier (2021.emnlp-main)

Challenge:	Experimental results show that a sequence-to-sequence learning framework with neural networks can be effective for Chinese Spelling Correction (CSC)
Approach:	They propose a sequence-to-sequence learning framework with neural networks that generates more valuable training instances and adds task-specific examples to enhance the model.
Outcome:	The proposed method improves generalization and robustness of multiple CSC models across three datasets.

Copied to clipboard

Liping Yuan, Xiaoqing Zheng, Yi Zhou, Cho-Jui Hsieh, Kai-Wei Chang

Weight Perturbation as Defense against Adversarial Word Substitutions (2022.findings-emnlp)

Challenge:	Existing studies show that deep neural networks are vulnerable to adversarial examples . a small perturbation to an input alters the model prediction .
Approach:	They propose a genetic algorithm to find models that can induce adversarial examples to fool models . they propose word replacement rules that can be used for model diagnostics from these examples .
Outcome:	The proposed model can fool almost all existing models, while ignoring the data bias in the training set.

Copied to clipboard

Jianhan Xu, Linyang Li, Jiping Zhang, Xiaoqing Zheng, Kai-Wei Chang, Cho-Jui Hsieh, Xuanjing Huang

TableVLM: Multi-modal Pre-training for Table Structure Recognition (2023.acl-long)

Challenge:	Existence and pervasiveness of textual adversarial examples have raised serious concerns to security-critical applications.
Approach:	They propose to perform weight perturbations in the parameter space rather than the input feature space to improve adversarial robustness of NLP models.
Outcome:	The proposed method improves adversarial robustness of models by performing weight perturbations in the parameter space rather than the input feature space.

Copied to clipboard

Leiyuan Chen, Chengsong Huang, Xiaoqing Zheng, Jinshu Lin, Xuanjing Huang

Multimodality

Challenge:	Tables are useful for displaying data in an organized manner, but they are difficult to extract from images because of their structure, notation, and representation.
Approach:	They propose a multi-modal pre-training model for table structure recognition that captures table structure-related features by multiple unsupervised objectives inspired by masked visual-language modeling.
Outcome:	The proposed model improves tree-editing-distance-score on ComplexTable by 1.97% .

Defense against Synonym Substitution-based Adversarial Attacks via Dirichlet Neighborhood Ensemble (2021.acl-long)

Copied to clipboard

Yi Zhou, Xiaoqing Zheng, Cho-Jui Hsieh, Kai-Wei Chang, Xuanjing Huang

UPLex: Fine-Grained Personality Control in Large Language Models via Unsupervised Lexical Modulation (2025.findings-emnlp)

Challenge:	Recent studies show vulnerability of deep neural networks to adversarial examples that intentionally fool the networks.
Approach:	They propose a method for training a robust model to defense synonym substitution-based attacks by sampling embedding vectors for each word in an input sentence and augmenting them with the training data.
Outcome:	The proposed method outperforms other proposed defense methods by a significant margin across different network architectures and multiple data sets.

Copied to clipboard

Tianlong Li, Wenhao Liu, Muling Wu, Shihan Dou, Zhenghua Wang, Changze Lv, Xiaohua Wang, Xiaoqing Zheng, Xuanjing Huang

Improving the Adversarial Robustness of NLP Models by Information Bottleneck (2022.findings-acl)

Challenge:	Personality is a crucial factor that shapes human communication patterns, thereby regulating the personalities of large language models (LLMs).
Approach:	They propose a method that uses an Unsupervisedly-Built Personalized Lexicon (UPL) during the decoding phase to manipulate LLM’s personality traits.
Outcome:	The proposed method can modulate the personality expression of large language models by dynamically altering their predicted probability of upcoming words in a pluggable fashion.

Copied to clipboard

Cenyuan Zhang, Xiang Zhou, Yixin Wan, Xiaoqing Zheng, Kai-Wei Chang, Cho-Jui Hsieh

Advancing Parameter Efficiency in Fine-tuning via Representation Editing (2024.acl-long)

Challenge:	Existing studies have shown that adversarial examples can be directly attributed to the presence of non-robust features.
Approach:	They propose to capture task-specific robust features while eliminating non-robust ones . they show that models can achieve significant improvement in robust accuracy .
Outcome:	The proposed method outperforms all defense methods on SST-2, AGNEWS and IMDB datasets while achieving no performance drop.

Copied to clipboard

Muling Wu, Wenhao Liu, Xiaohua Wang, Tianlong Li, Changze Lv, Zixuan Ling, Zhu JianHao, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang

Beyond Single Labels: Improving Conversational Recommendation through LLM-Powered Data Augmentation (2025.acl-long)

Challenge:	Parameter Efficient Fine-Tuning (PEFT) has gained significant attention for its ability to achieve competitive results while updating only a small subset of trainable parameters.
Approach:	They propose a new approach to fine-tuning neural models that scales and biases the representation produced at each layer.
Outcome:	The proposed approach reduces the number of trainable parameters by a factor of 25,700 compared to full parameter fine-tuning and by . 32 compared with LoRA.

Copied to clipboard

Haozhe Xu, Xiaohua Wang, Changze Lv, Xiaoqing Zheng

Responsible & Trustworthy NLP

Challenge:	Existing methods for enhancing recommendation quality face false negatives . only one "silly cop movie" is labeled as positive, leading to suboptimal recommendations .
Approach:	They propose a data augmentation framework that leverages an LLM-based semantic retriever to identify diverse and semantically relevant items and filter them by a relevance scorer to remove noisy candidates.
Outcome:	The proposed approach improves performance on two benchmark datasets and user simulators.

VIB-Probe: Detecting and Mitigating Hallucinations in Vision-Language Models via Variational Information Bottleneck (2026.acl-long)

Copied to clipboard

Feiran Zhang, Yixin Wu, Zhenghua Wang, Xiaohua Wang, Changze Lv, Xuanjing Huang, Xiaoqing Zheng

Multimodality

Challenge:	Existing hallucination detection methods rely on external verification tools . however, entanglement of visual-linguistic syntax and noise makes it difficult to detect hallucis .
Approach:	They propose a hallucination detection framework that leverages the Variational Information Bottleneck theory to detect hallucinic heads and to infer hallucication mitigation strategies.
Outcome:	The proposed framework outperforms baselines in hallucinations and noise detection environments.

Generating Responses with a Specific Emotion in Dialog (P19-1)

Copied to clipboard

Zhenqiao Song, Xiaoqing Zheng, Lu Liu, Mu Xu, Xuanjing Huang

Dialogue Systems & Conversational Agents

Challenge:	EmoDS can express emotions in both ways, but it is difficult to scale to large datasets.
Approach:	They propose an emotional dialog system that can express emotions in both ways . they use strong emotional words and neutral words to increase the intensity of emotions .
Outcome:	The proposed system performs better than baselines in BLEU, diversity and quality of emotional expression.

Enhancing Unsupervised Semantic Parsing with Distributed Contextual Representations (2023.findings-acl)

Copied to clipboard

Zixuan Ling, Xiaoqing Zheng, Jianhan Xu, Jinshu Lin, Kai-Wei Chang, Cho-Jui Hsieh, Xuanjing Huang

Mitigating Position Bias in Transformers via Layer-Specific Positional Embedding Scaling (2026.findings-acl)

Challenge:	Existing methods to learn models on corpus of pairs of sentences require labor-intensive annotation.
Approach:	They propose to leverage distributed contextual word and phrase representations pre-trained on unlabelled texts to deal with homonymy and polysemy.
Outcome:	The proposed model achieves better accuracy on question-answering and relation extraction tasks.

Copied to clipboard

Zhenghua Wang, Yiran Ding, Changze Lv, Yixin Wu, Tianlong Li, Zhibo Xu, Muling Wu, Tianyuan Shi, Shizheng Li, Qi Qian, Xuanjing Huang, Xiaoqing Zheng

Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspective (2025.coling-main)

Challenge:	Existing methods to address the "lost-in-the-middle" problem suffer from high latency or suboptimal hand-crafted scaling strategies.
Approach:	They propose a layer-specific positional embedding scaling method that assigns distinct scaling factors to each layer.
Outcome:	Experiments show that the proposed method mitigates positional attention bias and delivers consistent improvements across multiple long-context benchmarks.

Copied to clipboard

Tianlong Li, Zhenghua Wang, Wenhao Liu, Muling Wu, Shihan Dou, Changze Lv, Xiaohua Wang, Xiaoqing Zheng, Xuanjing Huang

Hallucination Detection for Generative Large Language Models by Bayesian Sequential Estimation (2023.emnlp-main)

Challenge:	Recent surge in jailbreaking attacks has revealed significant vulnerabilities in Large Language Models (LLMs) however, limited research into the underlying mechanisms that make LLMs vulnerable to such attacks has been conducted.
Approach:	They propose that LLMs' self-safeguarding capability is linked to specific activity patterns within their representation space.
Outcome:	The proposed models can be detected with a few pairs of contrastive queries, and the robustness can be manipulated by weakening or strengthening these patterns.

Copied to clipboard

Xiaohua Wang, Yuliang Yan, Longtao Huang, Xiaoqing Zheng, Xuanjing Huang

Watermarking PLMs on Classification Tasks by Combining Contrastive Learning with Weight Perturbation (2023.findings-emnlp)

Challenge:	Existing methods for detecting hallucinations require large numbers of observations to be retrieved, increasing response times.
Approach:	They propose a framework that leverages Bayesian sequential analysis to optimize the trade-off between costs and benefits during the hallucination detection process.
Outcome:	The proposed framework surpasses existing methods in efficiency and precision of hallucination detection.

Copied to clipboard

Chenxi Gu, Xiaoqing Zheng, Jianhan Xu, Muling Wu, Cenyuan Zhang, Chengsong Huang, Hua Cai, Xuanjing Huang

Mitigating Hallucinations in VLMs: Enhancing Visual Attention via Head-Wise Perturbation (2026.findings-acl)

Challenge:	Large pre-trained language models (PLMs) are highly valuable intellectual property due to their expensive training costs.
Approach:	They propose to embed backdoors that can be triggered by specific inputs into models by model watermarking.
Outcome:	The proposed method can be used to protect the intellectual property of large pre-trained language models without knowledge about downstream tasks.

Copied to clipboard

Zhenghua Wang, Yixin Wu, Feiran Zhang, Qi Qian, Changze Lv, Xuanjing Huang, Xiaoqing Zheng

Multimodality

Challenge:	Vision–Language Models (VLMs) have demonstrated strong capabilities in tasks that require joint understanding of text and images.
Approach:	They propose a strategy that incorporates head-wise attention perturbation via continuous multiplicative noise coupled with a visual-guided loss focused on vision-sensitive text tokens to promote a more balanced attention distribution.
Outcome:	The proposed approach outperforms baseline models on three benchmarks and consistently outperformed the baseline model.

TMATH A Dataset for Evaluating Large Language Models in Generating Educational Hints for Math Word Problems (2025.coling-main)

Copied to clipboard

Changyong Qi, Yuang Wei, Haoxin Xu, Longwei Zheng, Peiji Chen, Xiaoqing Gu

Towards Adversarially Robust Text Classifiers by Learning to Reweight Clean Examples (2022.findings-acl)

Challenge:	Large Language Models (LLMs) are increasingly being applied in education, showing significant potential in personalized instruction, student feedback, and intelligent tutoring systems (ITSs).
Approach:	They propose a dataset specifically designed to evaluate LLMs’ ability to generate high-quality hints for Math Word Problems.
Outcome:	The proposed dataset shows that LLMs can generate more accurate and contextually appropriate educational hints for math word problems without offering direct answers.

Copied to clipboard

Jianhan Xu, Cenyuan Zhang, Xiaoqing Zheng, Linyang Li, Cho-Jui Hsieh, Kai-Wei Chang, Xuanjing Huang

Measure Children’s Mindreading Ability with Machine Reading (2023.findings-emnlp)

Challenge:	Existing defense methods improve the adversarial robustness by making models adapt to training set augmented with some adversarials.
Approach:	They propose to introduce a reweighting mechanism to calibrate the training distribution to obtain robust models.
Outcome:	The proposed method minimizes the loss of validation set mixed with clean examples and adversarial ones in an online learning manner.

Copied to clipboard

Yuliang Yan, Xiaohua Wang, Xiang Zhou, Xiaoqing Zheng, Xuanjing Huang

Visual Data in NLP

Challenge:	Existing scoring models do not take the features of the stories and video clips into account when scoring, which will reduce the accuracy of the models.
Approach:	They propose to leverage the features extracted from stories and videos related to the questions being asked during the children’s mindreading evaluation.
Outcome:	The proposed framework agrees well with human experts on scores produced by the models.

TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021.acl-demo)

Copied to clipboard

Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang, Rui Zheng, Zexiong Pang, Qinzhuo Wu, Zhengyan Li, Chong Zhang, Ruotian Ma, Zichu Fei, Ruijian Cai, Jun Zhao, Xingwu Hu, Zhiheng Yan, Yiding Tan, Yuan Hu, Qiyuan Bian, Zhihua Liu, Shan Qin, Bolin Zhu, Xiaoyu Xing, Jinlan Fu, Yue Zhang, Minlong Peng, Xiaoqing Zheng, Yaqian Zhou, Zhongyu Wei, Xipeng Qiu, Xuanjing Huang

Improving Continual Pre-training Through Seamless Data Packing (2025.findings-acl)

Challenge:	Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction.
Approach:	They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack.
Outcome:	The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses.

Copied to clipboard

Ruicheng Yin, Xuan Gao, Changze Lv, Xiaohua Wang, Xiaoqing Zheng, Xuanjing Huang