Papers by Xin Lu

ScaleBox: Enabling High-Fidelity and Scalable Code Verification for Large Language Models (2026.acl-demo)

Challenge:	Existing research has demonstrated that the ability of large language models (LLMs) to generate humorous sentences is limited to producing 25 unique jokes.
Approach:	They propose a multi-stage curriculum preference learning framework to optimize both pun structure preferences and humor preferences by a Chinese Pun dataset.
Outcome:	The proposed method significantly outperforms baseline models on Chinese and English benchmark datasets.

Copied to clipboard

Jiasheng Zheng, Xin Zheng, Boxi Cao, Pengbo Wang, Zhengzhao Ma, Qiming Zhu, Jiazhen Jiang, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun

BOLT: Fast Energy-based Controlled Text Generation with Tunable Biases (2023.acl-short)

Challenge:	Existing code sandboxes fail to provide accurate verification and efficiency under high-concurrency workloads.
Approach:	They propose a high-fidelity code verification system that provides sandbox feedback for RL training and evaluation.
Outcome:	The proposed system outperforms heuristic-matching baselines on LiveCodeBench and training stability on high-concurrency workloads.

Copied to clipboard

Xin Liu, Muhammad Khalifa, Lu Wang

Sentiment Analysis

Challenge:	Energy-based models (EBMs) have gained popularity for controlled text generation due to their high applicability to a wide range of constraints.
Approach:	They propose a language model with tunable biases to adjust the language model’s output logits.
Outcome:	The proposed model maintains the generator’s autoregressive nature to assert a strong control on token-wise conditional dependencies and overall fluency, and converges faster.

Enhancing Language Model Factuality via Activation-Based Confidence Calibration and Guided Decoding (2024.emnlp-main)

Copied to clipboard

Xin Liu, Farima Fatahi Bayat, Lu Wang

AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments (2026.acl-long)

Challenge:	Existing methods to calibrate language models are limited in inference-time efficiency or fail to provide informative signals.
Approach:	They propose an activation-based calibration method, ActCab, which trains a linear layer on top of the LM’s last-layer activations.
Outcome:	The proposed method improves on five popular QA benchmarks and reduces the average expected calibration error (ECE) score by up to 39%.

Copied to clipboard

Zhiheng Xi, Dingwen Yang, Jiaqi Liu, Jixuan Huang, Honglin Guo, Baodai Huang, Tinggang Chen, Qi Zhang, Zhonghang Lu, Chenyu Liu, Jiajun Sun, Jiazheng Zhang, Dingwei Zhu, Xin Guo, Junzhe Wang, Zhihao Zhang, Yuming Yang, Junjie Ye, Minghe Gao, Dongrui Liu, Jiaming Ji, Guohao Li, Tao Gui, Qi Zhang, Xuanjing Huang

TextMixer: Mixing Multiple Inputs for Privacy-Preserving Inference (2023.findings-emnlp)

Challenge:	Existing benchmarks evaluate agents in simplified, idealized settings, relying on pre-packaged tool interfaces, overlooking critical steps, and assume inputs are clean and fully specified.
Approach:	They propose a framework that evaluates language agents in simplified, idealized settings . they show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 .
Outcome:	Experiments on 15 proprietary and open-source models show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 .

Copied to clipboard

Xin Zhou, Yi Lu, Ruotian Ma, Tao Gui, Qi Zhang, Xuanjing Huang

mCLIP: Multilingual CLIP via Cross-lingual Transfer (2023.acl-long)

Challenge:	Pre-trained language models (PLMs) are often deployed as cloud services, enabling users to upload textual data and perform inference remotely.
Approach:	They propose a privacy-preserving inference framework called MixPi which aims to obfuscate a user's private input by mixing it with multiple other inputs.
Outcome:	The proposed framework surpasses existing privacy-preserving methods on token and sentence classification tasks.

Copied to clipboard

Guanhua Chen, Lu Hou, Yun Chen, Wenliang Dai, Lifeng Shang, Xin Jiang, Qun Liu, Jia Pan, Wenping Wang

Multilinguality

Challenge:	Existing multilingual vision-language pretrained models are biased towards English due to the lack of sufficient non-English image-text pairs.
Approach:	They propose to train a retrieval-efficient dual-stream multilingual VLP model by aligning CLIP model and a multilingual text encoder through a novel Triangle Cross-modal Knowledge Distillation method.
Outcome:	Empirical results show that mCLIP achieves new state-of-the-art performance for both zero-shot and finetuned multilingual image-text retrieval tasks.

RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generation (2024.emnlp-demo)

Copied to clipboard

Qinyu Luo, Yining Ye, Shihao Liang, Zhong Zhang, Yujia Qin, Yaxi Lu, Yesai Wu, Xin Cong, Yankai Lin, Yingli Zhang, Xiaoyin Che, Zhiyuan Liu, Maosong Sun

VeriFact: Enhancing Long-Form Factuality Evaluation with Refined Fact Extraction and Reference Facts (2025.emnlp-main)

Challenge:	Xia et al., 2018) demonstrate that a large language model can generate and maintain high-quality code documentation.
Approach:	They propose a large language model powered open-source framework for generating, maintaining, and updating code documentation.
Outcome:	The proposed framework generates high-quality documentation for the entire project.

Copied to clipboard

Xin Liu, Lechen Zhang, Sheza Munir, Yiyang Gu, Lu Wang

Information Extraction & Text Mining

Challenge:	Prior work focuses on accuracy and precision, but factuality evaluation is difficult due to inter-sentence dependencies.
Approach:	They introduce a factuality evaluation framework to enhance fact extraction . they also introduce 'factRBench' that evaluates both precision and recall .
Outcome:	The proposed framework enhances fact extraction by identifying incomplete and missing facts . it also evaluates precision and recall in long-form models, whereas prior work focuses on precision.

LiteVL: Efficient Video-Language Learning with Enhanced Spatial-Temporal Modeling (2022.emnlp-main)

Copied to clipboard

Dongsheng Chen, Chaofan Tao, Lu Hou, Lifeng Shang, Xin Jiang, Qun Liu

Executing Natural Language-Described Algorithms with Large Language Models: An Investigation (2024.lrec-main)

Challenge:	Recent large-scale video-language pre-trained models have shown appealing performance on downstream tasks.
Approach:	They propose a video-text model that adapts a pre-trained image-language model into a text-based model without heavy pre-training.
Outcome:	The proposed model outperforms existing models on video-text retrieval and video question answering tasks without heavy pre-training.

Copied to clipboard

Xin Zheng, Qiming Zhu, Hongyu Lin, Yaojie Lu, Xianpei Han, Le Sun

CKnowEdit: A New Chinese Knowledge Editing Dataset for Linguistics, Facts, and Logic Error Correction in LLMs (2025.acl-long)

Challenge:	Recent advances in large language models (LLMs) have revolutionized the field of natural language processing and artificial intelligence, creating new SOTAs and reaching human-level language understanding performance on a series of tasks and benchmarks.
Approach:	They propose to use an algorithm test set sourced from Introduction to Algorithm to assess LLMs' code execution abilities.
Outcome:	The proposed model can execute programs described in natural language as long as no heavy numeric computation is involved.

Copied to clipboard

Jizhan Fang, Tianhe Lu, Yunzhi Yao, Ziyan Jiang, Xin Xu, Huajun Chen, Ningyu Zhang

Extending Phrase Grounding with Pronouns in Visual Dialogues (2022.emnlp-main)

Challenge:	CKnowEdit is the first-ever knowledge editing dataset designed to correct linguistic, factual, and logical errors in Large Language Models.
Approach:	They propose a Chinese knowledge editing dataset to correct linguistic, factual, and logical errors in Large Language Models.
Outcome:	The proposed dataset highlights the challenges that LLMs face in mastering Chinese . CKnowEdit can correct linguistic, factual, and logical errors in Chinese, the authors show .

Copied to clipboard

Panzhong Lu, Xin Zhang, Meishan Zhang, Min Zhang

Retrieve, Discriminate and Rewrite: A Simple and Effective Framework for Obtaining Affective Response in Retrieval-Based Chatbots (2021.findings-emnlp)

Challenge:	Conventional phrase grounding aims to localize noun phrases mentioned in a caption to their corresponding image regions.
Approach:	They extend the task by considering pronouns to include noun phrases and pronounos . they construct a dataset of phrase grounding with noun and pronom phrases to image regions .
Outcome:	Experiments show that pronouns are easier to ground than noun phrases . a baseline model with coreference information can significantly boost the grounding performance .

Copied to clipboard

Xin Lu, Yijian Tian, Yanyan Zhao, Bing Qin

Sentiment Analysis

Challenge:	Existing work on retrieval-based chatbots has low-quality affect response . Existing frameworks for obtaining affective response are based on Retrieve-and-Rerank .
Approach:	They propose a retrieval-based framework which provides affective response for retrieval chatbots by using a new discriminate-and-rewrite mechanism.
Outcome:	The proposed framework outperforms existing baselines and can guarantee the quality of the response and satisfy the affect label.

Structured Pruning for Efficient Generative Pre-trained Language Models (2023.findings-acl)

Copied to clipboard

Chaofan Tao, Lu Hou, Haoli Bai, Jiansheng Wei, Xin Jiang, Qun Liu, Ping Luo, Ngai Wong

HyperCL: A Contrastive Learning Framework for Hyper-Relational Knowledge Graph Embedding with Hierarchical Ontology (2024.findings-acl)

Challenge:	Large-scale generative Pre-trained Language Models (PLMs) are limited in their deployment in real-world applications.
Approach:	They propose to prune the feed-forward networks of generative pre-trained language models to smaller widths without designing extra operators.
Outcome:	The proposed method achieves 1.51x/6.96x inference speedup on GPU/CPU with 67% size reduction.

Copied to clipboard

Yuhuan Lu, Weijian Yu, Xin Jing, Dingqi Yang

JointCoder: Exploring Automated ICD Coding on Real-World Chinese EHRs with a Multi-Agent Framework (2026.acl-demo)

Challenge:	Existing studies neglect the ontology of knowledge Graph (KG) embeddings and suffer from the dominance issue of facts over ontologies.
Approach:	They propose a framework for hyper-relational KG embeddings that captures the hierarchical ontology and a concept-aware contrastive loss to alleviate the dominance issue.
Outcome:	The proposed framework improves on three real-world datasets and shows that it can integrate with other embedding methods and improve link prediction performance.

Copied to clipboard

Kangjun Liu, Zhenyu Li, Jianlei Wang, Hongjiao Guan, Ying Lian, Guoqiang Chen, Tao Xin, Wenpeng Lu

Challenge:	Existing automated ICD coding systems face several fundamental challenges due to the limited availability of publicly available Chinese ICD datasets.
Approach:	They propose to use a Chinese ICD coding dataset and a multi-agent framework to reformulate ICD as a joint disease-procedure coding task.
Outcome:	The proposed system outperforms state-of-the-art methods on real-world Chinese ICD coding datasets and 1.7B-parameter models.

Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language Models (2026.acl-long)

Copied to clipboard

Jia Deng, Junyi Li, Xin Zhao, Jinpeng Wang, Hongyu Lu, Ji-Rong Wen

An Iterative Emotion Interaction Network for Emotion Recognition in Conversations (2020.coling-main)

Challenge:	Existing methods for full-attention dLLMs rely on random masking strategies that overlook intrinsic token dependencies.
Approach:	They propose an attention-guided denoising and optimization framework that aligns training and optimization with attention-derived dependencies.
Outcome:	The proposed framework outperforms state-of-the-art methods on mathematical and coding benchmarks.

Copied to clipboard

Xin Lu, Yanyan Zhao, Yang Wu, Yijian Tian, Huipeng Chen, Bing Qin

Dialogue Systems & Conversational Agents

Challenge:	Emotion recognition in conversations (ERC) is a task that aims to recognize the emotion of each utterance in conversations.
Approach:	They propose an iterative emotion interaction network which uses iterativly predicted emotion labels instead of gold emotion labels to explicitly model the emotion interaction.
Outcome:	The proposed method retains state-of-the-art performance on two datasets and achieves high accuracy.

A Table-to-Text Framework with Heterogeneous Multidominance Attention and Self-Evaluated Multi-Pass Deliberation (2023.findings-emnlp)

Copied to clipboard

Xi Chen, Xinjiang Lu, Haoran Xin, Wenjun Peng, Haoyang Duan, Feihu Jiang, Jingbo Zhou, Hui Xiong

Structured Data in NLP

Challenge:	Table-to-text works have been widely applied in different domains, such as weather forecast and financial report generation.
Approach:	They propose a table-to-text approach on top of Self-evaluated multi-pass Generation and Heterogenous Multidominance Attention to explore the hierarchical structure.
Outcome:	The proposed method outperforms several SOTA methods quantitatively and qualitatively on three public datasets.

Compression of Generative Pre-trained Language Models via Quantization (2022.acl-long)

Copied to clipboard

Chaofan Tao, Lu Hou, Wei Zhang, Lifeng Shang, Xin Jiang, Qun Liu, Ping Luo, Ngai Wong

AgentCPM-GUI: Building Mobile-Use Agents with Reinforcement Fine-Tuning (2025.emnlp-demos)

Challenge:	Existing methods to compress generative pre-trained language models fail on generative tasks due to homogeneous word embeddings and limited memory.
Approach:	They propose a token-level contrastive distillation method to learn distinguishable word embeddings and a module-wise dynamic scaling method to make quantizers adaptive to different modules.
Outcome:	The proposed method outperforms the state-of-the-art compression methods on generative PLMs by a clear margin.

Copied to clipboard

Zhong Zhang, Yaxi Lu, Yikun Fu, Yupeng Huo, Shenzhi Yang, Yesai Wu, Han Si, Xin Cong, Haotian Chen, Yankai Lin, Xie Xie, Wei Zhou, Wang Xu, Zhou Su, Zhongwu Zhai, Xiaoming Liu, null Meiyudong, Jianming Xu, Hongyan Tian, Chongyi Wang, Chi Chen, Yuan Yao, Zhiyuan Liu, Maosong Sun

AgentGym: Evaluating and Training Large Language Model-based Agents across Diverse Environments (2025.acl-long)

Challenge:	Large language model agents have enabled GUI-based automation, but their deployment is limited by noisy data, poor generalization, and lack of support for non-English GUIs.
Approach:	They propose an 8B-parameter GUI agent built for robust and efficient on-device GUI interaction.
Outcome:	The proposed GUI agent achieves promising performance on five public benchmarks and proposed Chinese benchmark CAGUI.

Copied to clipboard

Zhiheng Xi, Yiwen Ding, Wenxiang Chen, Boyang Hong, Honglin Guo, Junzhe Wang, Xin Guo, Dingwen Yang, Chenyang Liao, Wei He, Songyang Gao, Lu Chen, Rui Zheng, Yicheng Zou, Tao Gui, Qi Zhang, Xipeng Qiu, Xuanjing Huang, Zuxuan Wu, Yu-Gang Jiang

Critic-CoT: Boosting the Reasoning Abilities of Large Language Model via Chain-of-Thought Critic (2025.findings-acl)

Challenge:	Large language models (LLMs) are promising foundations to build generally-capable agents . however, the community lacks a unified interactive framework that covers diverse environments for comprehensive evaluation of agents.
Approach:	They propose a framework that features 7 real-world scenarios, 14 environments, and 89 tasks for unified, real-time, and concurrent agent interaction.
Outcome:	The proposed framework features 7 real-world scenarios, 14 environments, and 89 tasks for unified, real-time, and concurrent agent interaction.

Copied to clipboard

Xin Zheng, Jie Lou, Boxi Cao, Xueru Wen, Yuqiu Ji, Hongyu Lin, Yaojie Lu, Xianpei Han, Debing Zhang, Le Sun

RepoGenesis: Benchmarking End-to-End Microservice Generation from Readme to Repository (2026.acl-long)

Challenge:	Existing approaches to improve the reasoning performance of large language models rely on intuitive instance-level feedback, which limits the reasoning capabilities.
Approach:	They propose a framework that pushes LLMs toward System-2-like critic capability by using a step-wise CoT reasoning paradigm and automatic construction of weak-supervision data without human annotation.
Outcome:	The proposed model significantly improves task-solving performance by filtering out invalid solutions or iterative refinement.

Copied to clipboard

Zhiyuan Peng, Xin Yin, Pu Zhao, Fangkai Yang, Lu Wang, Ran Jia, Xu Chen, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang

Challenge:	Existing benchmarks focus on isolated function/class-level generation, neglecting complete microservice repository generation.
Approach:	They propose a multilingual benchmark for repository-level end-to-end web microservice generation that reflects real-world development workflows.
Outcome:	The benchmark compared 106 repositories across 18 domains and 11 frameworks and 1,258 API endpoints and 2,335 test cases.

GhostBERT: Generate More Features with Cheap Operations for BERT (2021.acl-long)

Copied to clipboard

Zhiqi Huang, Lu Hou, Lifeng Shang, Xin Jiang, Xiao Chen, Qun Liu

LongHeads: Multi-Head Attention is Secretly a Long Context Processor (2024.findings-emnlp)

Challenge:	Existing studies show that some parameters in pre-trained language models can be pruned away without severe accuracy degradation.
Approach:	They propose a method which generates more features with very cheap operations from the remaining features and can be applied to unpruned BERT models to enhance their performance.
Outcome:	Empirical results on the GLUE benchmark on three backbone models (i.e., BERT, RoBERTa and ELECTRA) verify the efficacy of the proposed method.

Copied to clipboard

Yi Lu, Xin Zhou, Wei He, Jun Zhao, Tao Ji, Tao Gui, Qi Zhang, Xuanjing Huang

Green & Sustainable NLP

Challenge:	Large language models struggle to process lengthy inputs due to limited length generalization and attention’s quadratic computational demands.
Approach:	They propose a training-free framework that allows each head to attend to important context chunks instead of allowing each head a full sentence .
Outcome:	The proposed framework unlocks multi-head attention's untapped potential by allowing each head to attend to important context chunks instead of the full sentence.

Wukong-Reader: Multi-modal Pre-training for Fine-grained Visual Document Understanding (2023.acl-long)

Copied to clipboard

Haoli Bai, Zhiguang Liu, Xiaojun Meng, Li Wentao, Shuang Liu, Yifeng Luo, Nian Xie, Rongfu Zheng, Liangwei Wang, Lu Hou, Jiansheng Wei, Xin Jiang, Qun Liu

BinaryBERT: Pushing the Limit of BERT Quantization (2021.acl-long)

Challenge:	Existing solutions for visual document understanding lack granularity of document textlines.
Approach:	They propose a supervised pre-training program to leverage structural knowledge nested in document textlines to achieve fine-grained alignment between visual regions and texts.
Outcome:	The proposed system performs better on various VDU tasks in English and Chinese.

Copied to clipboard

Haoli Bai, Wei Zhang, Lu Hou, Lifeng Shang, Jin Jin, Xin Jiang, Qun Liu, Michael Lyu, Irwin King

PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generation (2022.findings-aacl)

Challenge:	Recent pre-trained language models have achieved remarkable performance improvement in various tasks, but the improvement generally comes at the cost of increasing model size and computation.
Approach:	They propose a binary quantization technique which initializes binaryBERT by splitting from a ternary network.
Outcome:	The proposed model achieves state-of-the-art performance on the GLUE and SQUAD benchmarks while being 24x smaller.

Copied to clipboard

Siqi Bao, Huang He, Fan Wang, Hua Wu, Haifeng Wang, Wenquan Wu, Zhihua Wu, Zhen Guo, Hua Lu, Xinxian Huang, Xin Tian, Xinchao Xu, Yingzhan Lin, Zheng-Yu Niu

Dialogue Systems & Conversational Agents

Challenge:	Experimental results show PLATO-XL achieves state-of-the-art results across multiple conversational tasks.
Approach:	They propose to train PLATO-XL models with up to 11 billion parameters, trained on Chinese and English social media conversations.
Outcome:	The proposed model achieves state-of-the-art on multiple conversational tasks, verifying its potential as a foundation model of conversational AI.

Beyond Full Fine-tuning: Harnessing the Power of LoRA for Multi-Task Instruction Tuning (2024.lrec-main)

Copied to clipboard

Chunlei Xin, Yaojie Lu, Hongyu Lin, Shuheng Zhou, Huijia Zhu, Weiqiang Wang, Zhongyi Liu, Xianpei Han, Le Sun

SelfAug: Mitigating Catastrophic Forgetting in Retrieval-Augmented Generation via Distribution Self-Alignment (2025.findings-emnlp)

Challenge:	Low-Rank Adaptation (LoRA) is a parameter-efficient fine-tuning algorithm for large-scale language models.
Approach:	They conduct a systematic study of Low-Rank Adaptation (LoRA) on diverse tasks and rich resources with different learning capacities.
Outcome:	The proposed algorithm can achieve remarkable performance in high-resource and multi-task scenarios, even comparable to full fine-tuning.

Copied to clipboard

Yuqing Huang, Rongyang Zhang, Qimeng Wang, Chengqiang Lu, Yan Gao, null Yiwu, Yao Hu, Xuyang Zhi, Guiquan Liu, Xin Li, Hao Wang, Enhong Chen

PychoAgent: Psychology-driven LLM Agents for Explainable Panic Prediction on Social Media during Sudden Disaster Events (2025.emnlp-main)

Challenge:	Existing solutions for supervised fine-tuning often lead to catastrophic forgetting, where models lose their previously acquired knowledge and general capabilities.
Approach:	They propose a self-distribution alignment method that aligns input sequence logits to preserve the model’s semantic distribution, thereby mitigating catastrophic forgetting and improving downstream performance.
Outcome:	The proposed method achieves a superior balance between downstream learning and general capability retention.

Copied to clipboard

Mengzhu Liu, Zhengqiu Zhu, Chuan Ai, Chen Gao, Xinghong Li, Lingnan He, Kaisheng Lai, Yingfeng Chen, Xin Lu, Yong Li, Quanjun Yin

Explainability & Interpretability in NLP

Challenge:	Social media's rich information content and spatiotemporal granularity provide unique opportunities for emotion prediction and management.
Approach:	They propose a Psychology-driven generative Agent framework for explainable panic prediction based on emotion arousal theory.
Outcome:	The proposed framework improves panic emotion prediction performance by 13% to 21% compared to baseline models.

Aligning Retrieval with Reader Needs: Reader-Centered Passage Selection for Open-Domain Question Answering (2025.coling-main)

Copied to clipboard

Chunlei Xin, Shuheng Zhou, Xuanang Chen, Yaojie Lu, Huijia Zhu, Weiqiang Wang, Zhongyi Liu, Xianpei Han, Le Sun

Information Retrieval

Challenge:	Existing retrieval methods aim to gather relevant passages but fail to prioritize consistent and useful information for the reader.
Approach:	They propose a novel method which re-ranks passages based on the reader's prediction probability distribution and clusters passage according to the predicted answers.
Outcome:	The proposed method improves the quality of evidence passages under zero-shot scenarios.

Don’t Lose Yourself! Empathetic Response Generation via Explicit Self-Other Awareness (2023.findings-acl)

Copied to clipboard

Weixiang Zhao, Yanyan Zhao, Xin Lu, Bing Qin

Text Generation

Challenge:	Existing attempts to generate empathy with other-awareness ignore to include self-a awareness to consider the own views of the self in their responses.
Approach:	They propose to include self-awareness to consider the own views of the self in empathetic response generation by integrating three stages of self-other awareness into the process.
Outcome:	The proposed method is superior to existing methods on the benchmark dataset.

Sailor: Open Language Models for South-East Asia (2024.emnlp-demo)

Copied to clipboard

Longxu Dou, Qian Liu, Guangtao Zeng, Jia Guo, Jiahui Zhou, Xin Mao, Ziqi Jin, Wei Lu, Min Lin

Is the Attention Matrix Really the Key to Self-Attention in Multivariate Long-Term Time Series Forecasting? (2026.acl-long)

Challenge:	Large language models (LLMs) rely on English data for training, but are often not comparable across other languages.
Approach:	They propose to develop a family of open language models for SEA languages . they use BPE dropout, aggressive data cleaning and deduplication to improve model robustness .
Outcome:	The proposed models perform well across four benchmarks, including commonsense reasoning, question answering, reading comprehension and examination.

Copied to clipboard

Xinyu Li, Kexi Chen, Jiajie Shen, Ying Zheng, Hong Lu, Jin Zhao, Xin Wang

Separate the Wheat from the Chaff: A Post-Hoc Approach to Safety Re-Alignment for Fine-Tuned Language Models (2025.findings-acl)

Challenge:	In multivariate long-term time series forecasting, it is widely believed that the effectiveness of self-attention arises from its attention matrix.
Approach:	They propose a multi-branch MLP that isolates the ‘multi-brain mapping with element-wise operation’ structure from the Transformer and shows that it achieves competitive performance.
Outcome:	The proposed model outperforms three classic and three latest Transformer models and shows that it achieves competitive performance.

Copied to clipboard

Di Wu, Xin Lu, Yanyan Zhao, Bing Qin

Multimodal Procedural Planning via Dual Text-Image Prompting (2024.findings-emnlp)

Challenge:	Large language models achieve effective safety alignment at the time of release, but fine-tuning often compromises safety mechanisms.
Approach:	They propose a method that performs safety realignment for large language models . they identify unsafe delta parameters from the fine-tuned models and recalibrate the retained parameters .
Outcome:	The proposed method improves safety performance on safety benchmarks and jailbreak attacks while maintaining their performance on downstream tasks.

Copied to clipboard

Yujie Lu, Pan Lu, Zhiyu Chen, Wanrong Zhu, Xin Wang, William Yang Wang

YuLan-Mini: Pushing the Limits of Open Data-efficient Language Model (2025.acl-long)

Challenge:	Embodied agents have demonstrated performance in following instructions informed by texts and images . however, the potential of models providing useful guidelines for humans to complete tasks remains underexplored .
Approach:	They propose a multimodal procedural planning task that generates paired text-image plans . this task provides more complementary and informative guidance than unimodal plans a . authors propose modality prompting methods that leverage zero-shot reasoning ability .
Outcome:	The proposed method improves the interaction in dual modalities and provides more information than unimodal plans.

Copied to clipboard

Hu Yiwen, Huatong Song, Jie Chen, Jia Deng, Jiapeng Wang, Kun Zhou, Yutao Zhu, Jinhao Jiang, Zican Dong, Yang Lu, Xu Miao, Xin Zhao, Ji-Rong Wen

Imagination-Augmented Natural Language Understanding (2022.naacl-main)

Challenge:	prevailing pre-training approaches for large language models involve several complexities.
Approach:	They propose a low-cost training recipe and a robust optimization approach to mitigate training instability . they also propose synthesis, curriculum, and data selection pipelines to integrate data .
Outcome:	The proposed model achieves top-tier performance among models with similar parameter scale . it is comparable to industry-leading models that require significantly more data .

Copied to clipboard

Yujie Lu, Wanrong Zhu, Xin Wang, Miguel Eckstein, William Yang Wang

Answer Convergence as a Signal for Early Stopping in Reasoning (2025.emnlp-main)

Challenge:	Existing methods for Natural Language Understanding focus on textual signals, which hinders models from learning efficiently from limited data samples.
Approach:	They propose an Imagination-Augmented Cross-modal Encoder to solve natural language understanding tasks from a novel learning perspective.
Outcome:	The proposed learning paradigm bridges the gap between human and agent language understanding in both linguistic and perceptual procedures.

Copied to clipboard

Xin Liu, Lu Wang

AgentRM: Enhancing Agent Generalization with Reward Modeling (2025.acl-long)

Challenge:	a systematic study suggests that chain-of-thought prompting is unnecessary for producing correct answers.
Approach:	They propose three inference-time strategies to improve model efficiency by boosting end-of-reasoning signals and early stopping . they propose a method that learns when to stop based on internal activations .
Outcome:	The proposed methods reduce token usage with little or no accuracy drop on natural questions . the proposed methods also reduce tokens by over 40% on naturalquestions .

Copied to clipboard

Yu Xia, Jingru Fan, Weize Chen, Siyu Yan, Xin Cong, Zhong Zhang, Yaxi Lu, Yankai Lin, Zhiyuan Liu, Maosong Sun

Robustness in NLP

Challenge:	Existing LLM-based agents have strong performance on held-in tasks, but their generalizability to unseen tasks remains poor.
Approach:	They propose a reward-based generalizable reward model to guide the policy model for effective test-time search.
Outcome:	The proposed agentRM outperforms existing agents on held-in tasks by 8.8 points on average.

SeqAR: Jailbreak LLMs with Sequential Auto-Generated Characters (2025.naacl-long)

Copied to clipboard

Yan Yang, Zeguan Xiao, Xin Lu, Hongru Wang, Xuetao Wei, Hailiang Huang, Guanhua Chen, Yun Chen

Logit Arithmetic Elicits Long Reasoning Capabilities Without Training (2026.findings-acl)

Challenge:	Existing studies have focused on the potential misuse of large language models (LLMs) however, the ability to align LLMs with human values is still vulnerable to malicious attacks.
Approach:	They propose a red-teaming strategy to enhance LLM safety by using a framework to design jailbreak prompts automatically.
Outcome:	The proposed framework achieves attack success rates of 88% and 60% in cold-start scenarios.

Copied to clipboard

Yunxiang Zhang, Muhammad Khalifa, Lechen Zhang, Xin Liu, Ayoung Lee, Xinliang Frederick Zhang, Farima Fatahi Bayat, Lu Wang

TextFusion: Privacy-Preserving Pre-trained Model Inference via Token Fusion (2022.emnlp-main)

Challenge:	Large reasoning models exhibit long chain-of-thought reasoning with complex strategies such as backtracking and self-verification, yet, these capabilities typically require resource-intensive post-training.
Approach:	They propose a decoding-time approach which transfers long chain-of-thought reasoning capabilities from a substantially smaller reasoning guider to a large non-reasoning target.
Outcome:	The proposed method improves performance over a model 21x smaller than the target model by 21.5% and 24.2% over the model.

Copied to clipboard

Xin Zhou, Jinzhu Lu, Tao Gui, Ruotian Ma, Zichu Fei, Yuran Wang, Yong Ding, Yibo Cheung, Qi Zhang, Xuanjing Huang

ShortGPT: Layers in Large Language Models are More Redundant Than You Expect (2025.findings-acl)

Challenge:	Existing methods to preserve inference privacy are available as cloud services . however, the risk of privacy leakage remains, according to recent studies .
Approach:	They propose a method to preserve inference privacy by fusing token representations in the cloud.
Outcome:	The proposed method preserves inference privacy without sacrificing performance on different scenarios.

Copied to clipboard

Xin Men, Mingyu Xu, Qingyu Zhang, Qianhao Yuan, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, Weipeng Chen

BehaviorSFT: Behavioral Token Conditioning for Health Agents Across the Proactivity Spectrum (2025.findings-emnlp)

Challenge:	Recent studies have identified significant redundancy in large language models . quantization and pruning are two methods that reduce computational resources .
Approach:	They propose simple pruning methods that prune redundant layers based on their BI scores.
Outcome:	The proposed pruning methods demonstrate superior performance over previous pruning methods.

Copied to clipboard

Yubin Kim, Zhiyuan Hu, Hyewon Jeong, Eugene W Park, Shuyue Stella Li, Chanwoo Park, Shiyun Xiong, MingYu Lu, Hyeonhoon Lee, Xin Liu, Daniel McDuff, Cynthia Breazeal, Samir Tulebaev, Hae Won Park

Challenge:	Large Language Models (LLMs) struggle with proactive engagement, authors say . a blind clinical evaluation confirmed that trained agents exhibit more realistic clinical behavior .
Approach:	They propose a training strategy using behavioral tokens to explicitly condition LLMs for dynamic behavioral selection.
Outcome:	The proposed training strategy boosts performance on both benchmarks.

From Generalist to Specialist: A Survey of Large Language Models for Chemistry (2025.coling-main)

Copied to clipboard

Yang Han, Ziping Wan, Lu Chen, Kai Yu, Xin Chen

Enhanced Language Model Truthfulness with Learnable Intervention and Uncertainty Expression (2024.findings-acl)

Challenge:	Existing studies on pretraining of LLMs on extensive web-based texts are insufficient for advanced scientific discovery, especially in chemistry.
Approach:	They outline methodologies for incorporating domain-specific chemistry knowledge and multi-modal information into LLMs and conceptualize chemistry LLM agents using chemistry tools.
Outcome:	The proposed models are based on domain-specific chemistry knowledge and multi-modal information and are capable of accelerating scientific research.

Copied to clipboard

Farima Fatahi Bayat, Xin Liu, H. Jagadish, Lu Wang

UER: An Open-Source Toolkit for Pre-training Models (D19-3)

Challenge:	Large language models (LLMs) generate long-form and coherent text, yet they often hallucinate facts, which undermines their reliability.
Approach:	They propose a Learnable Intervention method for Truthfulness Optimization that automatically identifies the optimal intervention intensity tailored to each query context.
Outcome:	Experiments on multiple LLMs and question-answering datasets show that LITO improves truthfulness while preserving task accuracy.

Copied to clipboard

Zhe Zhao, Hui Chen, Jinbin Zhang, Xin Zhao, Tao Liu, Wei Lu, Xi Chen, Haotang Deng, Qi Ju, Xiaoyong Du

Retrieved Sequence Augmentation for Protein Representation Learning (2024.emnlp-main)

Challenge:	Existing work on pre-training models have shown that it is important to use a framework to deploy various pre- training models efficiently.
Approach:	They propose an assemble-on-demand pre-training toolkit that assembles pre-trained models on demand and encapsulates them with rich modules.
Outcome:	The proposed framework can reproduce state-of-the-art models or develop models that remain unexplored.

Copied to clipboard

Chang Ma, Haiteng Zhao, Lin Zheng, Jiayi Xin, Qintong Li, Lijun Wu, Zhihong Deng, Yang Lu, Qi Liu, Sheng Wang, Lingpeng Kong

Experience-Driven Reflective Co-Evolution of Prompts and Heuristics for Autonomous Algorithm Design (2026.findings-acl)

Challenge:	Using multiple sequence alignments (MSA) to extract evolutionary knowledge is limited.
Approach:	They propose to use multiple sequence alignments to augment protein representations . they propose to employ Retrieved Sequence Augmentation to enhance protein representation learning .
Outcome:	The proposed method surpasses MSA Transformer by 5% in structural and property prediction tasks while being 373 times faster.

Copied to clipboard

Yihong Liu, Junyi Li, Hongyu Lu, Xin Zhao, Ji-Rong Wen

TernaryBERT: Distillation-aware Ultra-low Bit BERT (2020.emnlp-main)

Challenge:	Combinatorial optimization has long been dominated by manually engineered heuristics, which require substantial expert intuition and implementation overhead.
Approach:	They propose a framework that couples an island migration model with elite selection to maintain population diversity.
Outcome:	The proposed framework achieves superior accuracy on the Traveling Salesman and Bin Packing Problems.

Copied to clipboard

Wei Zhang, Lu Hou, Yichun Yin, Lifeng Shang, Xiao Chen, Xin Jiang, Qun Liu

MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation (2025.emnlp-main)

Challenge:	Transformer-based pre-training models like BERT are computationally expensive and limited to resource-constrained devices.
Approach:	They propose a method which ternarizes the weights in a fine-tuned BERT model.
Outcome:	The proposed method outperforms the other methods on the GLUE and SQUAD benchmarks while being 14.9x smaller.

Copied to clipboard

Weihao Xuan, Rui Yang, Heli Qi, Qingcheng Zeng, Yunze Xiao, Aosong Feng, Dairui Liu, Yun Xing, Junjue Wang, Fan Gao, Jinghui Lu, Yuang Jiang, Huitao Li, Xin Li, Kunyu Yu, Ruihai Dong, Shangding Gu, Yuekang Li, Xiaofei Xie, Felix Juefei-Xu, Foutse Khomh, Osamu Yoshie, Qingyu Chen, Douglas Teodoro, Nan Liu, Randy Goebel, Lei Ma, Edison Marrese-Taylor, Shijian Lu, Yusuke Iwasawa, Yutaka Matsuo, Irene Li

Multilinguality

Challenge:	Existing large language model evaluation benchmarks focus on English, while current multilingual tasks lack parallel questions that specifically assess cross-lingual reasoning abilities.
Approach:	They propose a comprehensive benchmark covering 29 languages, built on an English benchmark.
Outcome:	The MMLU-ProX is a comprehensive benchmark covering 29 languages, built on an English benchmark.

Collaborative Generative AI: Integrating GPT-k for Efficient Editing in Text-to-Image Generation (2023.emnlp-main)

Copied to clipboard

Wanrong Zhu, Xinyi Wang, Yujie Lu, Tsu-Jui Fu, Xin Wang, Miguel Eckstein, William Wang

Knowing When to Quit: Diagnosing and Training LLMs to Abort Futile Reasoning (2026.findings-acl)

Challenge:	Experimental results show that GPT-k models focus more on inserting modifiers than predicting spontaneous changes in the primary subject matter.
Approach:	They compare the common edits made by humans and GPT-k models to examine their performance in prompting T2I.
Outcome:	The proposed models improve the prompt editing process by 20-30%, the authors show . they show that humans tend to replace words and phrases with modifiers .

Copied to clipboard

Xinyan Guan, Jiali Zeng, Chunlei Xin, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun, Fandong Meng

HyperFM: Fact-Centric Multimodal Fusion for Link Prediction over Hyper-Relational Knowledge Graphs (2025.acl-long)

Challenge:	Large language models generate costly yet semantically void reasoning on beyond-capability tasks . the dominant failure mode is specious reasoning, superficially valid outputs with subtle hallucinations .
Approach:	They propose a capability-aligned reinforcement learning approach that aligns model behavior with capability boundaries.
Outcome:	The proposed model reduces futile reasoning while maintaining performance across tasks.

Copied to clipboard

Yuhuan Lu, Weijian Yu, Xin Jing, Dingqi Yang

Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation (2022.findings-acl)

Challenge:	Existing link prediction techniques focus on learning the complex relationships between entities and relations while ignoring the multimodal information.
Approach:	They propose a fact-centric fusion technique that captures complex interactions between different data modalities while accommodating the hyper-relational structure of the KG in a facts-centric manner.
Outcome:	The proposed technique improves on two real-world KG datasets by 6.0-6.8% over baselines.

Copied to clipboard

Wenliang Dai, Lu Hou, Lifeng Shang, Xin Jiang, Qun Liu, Pascale Fung

Learning to Generate Structured Output with Schema Reinforcement Learning (2025.acl-long)

Challenge:	Recent large-scale vision-language pre-training models are powerful in multimodal classification and retrieval tasks.
Approach:	They propose to augment a vision-language pre-training model with a textual pre-trained language model . the model achieves 44.5% zero-shot accuracy on multimodal generation tasks .
Outcome:	The proposed model achieves 44.5% zero-shot accuracy on open-ended visual question answering and image captioning tasks.

Copied to clipboard

Yaxi Lu, Haolun Li, Xin Cong, Zhong Zhang, Yesai Wu, Yankai Lin, Zhiyuan Liu, Fangming Liu, Maosong Sun

Personalized Question Answering with User Profile Generation and Compression (2025.findings-emnlp)

Challenge:	Recent advances in large language models have facilitated the development of intelligent applications like automatic web search (Qin et al., 2023) Several methods exist for generating JSON strings from LLMs, including Prompting but often miss certain schemas.
Approach:	They propose to use 40K different JSON schemas to assess models' ability to generate valid JSON outputs.
Outcome:	The proposed model improves both in generating JSON outputs and downstream tasks.

Copied to clipboard

Hang Su, Yun Yang, Tianyang Liu, Xin Liu, Peng Pu, Xuesong Lu

Natural Language Interfaces

Challenge:	Large language models are prone to providing “midguy” answers regardless of users’ knowledge background, thereby failing to meet each user’s personalized needs.
Approach:	They propose to generate personalized answers with LLMs based on users’ past question-answering records.
Outcome:	The proposed method generates personalized answers based on user's past question-answering records.

Visualize Before You Write: Imagination-Guided Open-Ended Text Generation (2023.findings-eacl)

Copied to clipboard

Wanrong Zhu, An Yan, Yujie Lu, Wenda Xu, Xin Wang, Miguel Eckstein, William Yang Wang

Making Harmful Behaviors Unlearnable for Large Language Models (2024.findings-acl)

Challenge:	Existing tools for text-to-image synthesis can visualize machine imaginations for a given context.
Approach:	They propose a framework that uses machine-generated images to guide language models in open-ended text generation.
Outcome:	The proposed framework is effective on open-ended text generation tasks while showing minor degeneration.

Copied to clipboard

Xin Zhou, Yi Lu, Ruotian Ma, Yujian Wei, Tao Gui, Qi Zhang, Xuanjing Huang

Chain-of-Rewrite: Aligning Question and Documents for Open-Domain Question Answering (2024.findings-emnlp)

Challenge:	Large language models (LLMs) are often customized by fine-tuning for the requirements of different domains.
Approach:	They propose a controllable training framework to make undesired behaviors unlearnable during the fine-tuning process.
Outcome:	The proposed framework makes undesired behaviors unlearnable during the fine-tuning process while preserving the ability to learn other information.

Copied to clipboard

Chunlei Xin, Yaojie Lu, Hongyu Lin, Shuheng Zhou, Huijia Zhu, Weiqiang Wang, Zhongyi Liu, Xianpei Han, Le Sun

Natural Language Interfaces

Challenge:	Existing approaches to answer open-domain question have encountered term mismatch and limited interaction between IR systems and large language models.
Approach:	They propose a method which leverages the guidance and feedback gained from the analysis to provide faithful and consistent extensions for effective question answering.
Outcome:	Experiments on four open-domain question answering datasets show the proposed method performs well under zero-shot settings.

Not All Metrics Are Guilty: Improving NLG Evaluation by Diversifying References (2024.naacl-long)

Copied to clipboard

Tianyi Tang, Hongyuan Lu, Yuchen Jiang, Haoyang Huang, Dongdong Zhang, Xin Zhao, Tom Kocmi, Furu Wei

Text Generation

Challenge:	Existing evaluation benchmarks with limited references may not accurately reflect the quality of the model’s hypotheses.
Approach:	They propose a method to enrich evaluation benchmarks by diversifying the expression of a single reference into multiple high-quality ones to cover the semantic space of the reference sentence as much as possible.
Outcome:	The proposed method can enhance evaluation benchmarks by diversifying the expression of reference into multiple high-quality ones to cover the semantic space of the reference sentence as much as possible.

Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios (2024.findings-acl)

Copied to clipboard

Shijue Huang, Wanjun Zhong, Jianqiao Lu, Qi Zhu, Jiahui Gao, Weiwen Liu, Yutai Hou, Xingshan Zeng, Yasheng Wang, Lifeng Shang, Xin Jiang, Ruifeng Xu, Qun Liu

Beyond Itinerary Planning—A Real-World Benchmark for Multi-Turn and Tool-Using Travel Tasks (2026.acl-long)

Challenge:	Existing benchmarks focus on simple synthesized queries that do not reflect real-world complexity, thereby offering limited perspectives in evaluating tool utilization.
Approach:	They propose a benchmark to evaluate LLMs’ ability in tool utilization within real-world scenarios.
Outcome:	The proposed benchmark improves LLMs’ ability in tool utilization within real-world scenarios and eliminates the restriction of pre-defined toolset.

Copied to clipboard

Xiang Cheng, Yulan Hu, Xiangwen Zhang, Lu Xu, Lide Tan, Zheng Pan, Xin Li, Yong Liu

JobFair: A Framework for Benchmarking Gender Hiring Bias in Large Language Models (2024.findings-emnlp)

Challenge:	Existing studies on LLM performance on travel planning have shown that existing settings are limited due to limited domain coverage, insufficient modeling of users’ implicit preferences in multi-turn conversations, and a lack of evaluation of agents’ capability boundaries.
Approach:	They propose a benchmark to evaluate LLMs' planning and tool-use abilities in real-world settings by collecting user queries, user preferences, and tools from real scenarios.
Outcome:	The proposed benchmark evaluates agents' capabilities in real-world settings and shows that even advanced models exhibit imbalanced performance across different capabilities.

Copied to clipboard

Ze Wang, Zekun Wu, Xin Guan, Michael Thaler, Adriano Koshiyama, Skylar Lu, Sachin Beepath, Ediz Ertekin, Maria Perez-Ortiz

Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Tasks (2026.acl-long)

Challenge:	a framework for benchmarking hierarchical gender hiring bias in Large Language Models (LLMs) is developed to protect vulnerable demographic groups.
Approach:	They propose a framework for benchmarking hierarchical gender hiring bias in Large Language Models for resume scoring.
Outcome:	The proposed framework reveals significant issues of reverse gender hiring bias and overdebiasing in ten state-of-the-art LLMs.

Copied to clipboard

Wenqi Zhang, Mengna Wang, Gangao Liu, Huixin Xu, Yiwei Jiang, Yongliang Shen, Guiyang Hou, Zhe Zheng, Hang Zhang, Xin Li, Jiajun Liu, Weiming Lu, Peng Li, Yueting Zhuang

Self-Foveate: Enhancing Diversity and Difficulty of Synthesized Instructions from Unsupervised Text via Multi-Level Foveation (2025.findings-acl)

Challenge:	Recent advances in reasoning models have demonstrated remarkable capabilities on mathematical and coding tasks, but their effectiveness in embodied domains remains largely unexplored.
Approach:	They propose a reasoning model for interactive embodied tasks that synthesizes 9.3k coherent Observation-Thought-Action trajectories containing 64k ego-centric images and 90k diverse reasoning processes.
Outcome:	The proposed model outperforms existing visual reasoning models by +9%, 24%, and +13% on long-horizon tasks.

Copied to clipboard

Mingzhe Li, Xin Lu, Yanyan Zhao

Sparse Latents Steer Retrieval-Augmented Generation (2025.acl-long)

Challenge:	Existing methods for training large language models rely on human effort for data annotation.
Approach:	They propose an unsupervised method that generates unsupervised instruction from unsupervised text using a "Micro-Scatter-Macro" method that excavates fine-grained information embedded in unsupervised texts.
Outcome:	The proposed method improves diversity and difficulty of synthesized instructions across multiple unsupervised corpora and diverse model architectures.

Copied to clipboard

Chunlei Xin, Shuheng Zhou, Huijia Zhu, Weiqiang Wang, Xuanang Chen, Xinyan Guan, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun

Information Retrieval

Challenge:	In this study, we uncover interpretable latents that govern RAG behavior in large language models . Sparse Autoencoders are used to control large language model (LLM) behavior .
Approach:	They leverage Sparse Autoencoders within the LLaMA Scope to uncover latents that govern RAG behaviors.
Outcome:	The proposed model can be used to control large language models without architectural modifications.

The BQ Corpus: A Large-scale Domain-specific Chinese Corpus For Sentence Semantic Equivalence Identification (D18-1)

Copied to clipboard

Jing Chen, Qingcai Chen, Xin Liu, Haijun Yang, Daohe Lu, Buzhou Tang

Information Extraction & Text Mining

Challenge:	Bank Question corpus is a corpus for sentence semantic equivalence identification (SSEI) because of rich expressions in natural languages, SSEI is really a challenging task.
Approach:	They propose to cluster 120,000 question pairs from 1-year online bank custom service logs into stacks by the Word Mover’s Distance (WMD) based Affinity Propagation algorithm to achieve questions with the same intent.
Outcome:	The proposed method achieves questions with the same intent by clustering deduplicated questions into stacks by the Word Mover’s Distance (WMD) based Affinity Propagation (AP) algorithm.

Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub (2025.acl-long)

Copied to clipboard

Bohan Lyu, Xin Cong, Heyang Yu, Pan Yang, Cheng Qian, Zihe Wang, Yujia Qin, Yining Ye, Yaxi Lu, Chen Qian, Zhong Zhang, Yukun Yan, Yankai Lin, Zhiyuan Liu, Maosong Sun

SDD: Self-Degraded Defense against Malicious Fine-tuning (2025.acl-long)

Challenge:	Existing approaches lack flexibility to address diverse and ever-evolving user queries in open domains.
Approach:	They propose to evaluate LLMs on open-domain knowledge that requires tools to solve diverse and ever-evolving user queries.
Outcome:	The proposed system outperforms baselines in the open domain task-solving benchmark.

Copied to clipboard

ZiXuan Chen, Weikai Lu, Xin Lin, Ziqian Zeng

TextObfuscator: Making Pre-trained Language Model a Privacy Protector via Obfuscating Word Representations (2023.findings-acl)

Challenge:	Open-source Large Language Models (LLMs) employ safety alignment methods to resist harmful instructions, but malicious fine-tuning can easily bypass these safeguards.
Approach:	They propose a framework to prevent malicious fine-tuning of large language models on harmful data by using alignment methods that encourage them to produce irrelevant responses to harmful prompts.
Outcome:	The proposed framework reduces the general capability of the LLM when malicious fine-tuning fails, rendering it incapable of following harmful instructions.

Copied to clipboard

Xin Zhou, Yi Lu, Ruotian Ma, Tao Gui, Yuran Wang, Yong Ding, Yibo Zhang, Qi Zhang, Xuanjing Huang