Papers by Xuanli He
Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineering (2025.naacl-long)
Copied to clipboard
Yu Zhao, Alessio Devoto, Giwon Hong, Xiaotang Du, Aryo Pradipta Gema, Hongru Wang, Xuanli He, Kam-Fai Wong, Pasquale Minervini
| Challenge: | Large language models store factual knowledge in their parameters but their parametric knowledge can conflict with the information provided in the context. |
| Approach: | They propose a training-free representation engineering method that uses pre-trained sparse auto-encoders to control the knowledge selection behaviour of large language models. |
| Outcome: | The proposed method can control the use of both knowledge sources to resolve knowledge conflict in open-domain question-answering tasks surpassing existing representation engineering methods (+10%) and contrastive decoding methods (+5%). |
PiCSAR: Probabilistic Confidence Selection and Ranking for Reasoning Chains (2026.findings-acl)
Copied to clipboard
Joshua Ong Jun Leang, Zheng Zhao, Aryo Pradipta Gema, Sohee Yang, Wai-Chung Kwan, Xuanli He, Wenda Li, Pasquale Minervini, Eleonora Giunchiglia, Shay B Cohen
| Challenge: | Recent studies show that large reasoning models (LLMs) achieve strong performance on complex reasoning tasks. |
| Approach: | They propose a method that scores each candidate generation using the joint log-likelihood of the reasoning and final answer. |
| Outcome: | The proposed method outperforms baselines with 2x fewer samples in 20 out of 25 comparisons. |
Mitigating Backdoor Poisoning Attacks through the Lens of Spurious Correlation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Modern NLP models are often trained over large untrustworthy datasets, raising the potential for a malicious adversary to compromise model behaviour. |
| Approach: | They propose to mitigate spurious correlations between textual triggers and classification labels by combining them with insertion-based attacks. |
| Outcome: | The proposed defence significantly reduces attack success rates across backdoor attacks and provides a near-perfect defence against insertion-based attacks. |
Differentially Private Representation for NLP: Formal Guarantee and An Empirical Study on Privacy and Fairness (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to learn text representations can encode private information of the input, thus can be exploited to recover such information with reasonable accuracy. |
| Approach: | They propose a novel approach to preserve privacy of the extracted representation from text by combining differential privacy with dropout. |
| Outcome: | The proposed approach preserves privacy of the extracted representation from text while masking words via dropout can enhance privacy. |
Using Natural Language Explanations to Improve Robustness of In-context Learning (2024.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that large language models excel in many tasks via in-context learning (ICL). However, ICL struggles to execute complex tasks such as arithmetic, commonsense, and symbolic reasoning. |
| Approach: | They propose to augment ICL with natural language explanations (NLEs) to produce further NLEs on adversarial datasets. |
| Outcome: | The proposed approach yields more accurate results than zero-shot-ICL and using only human-generated NLEs on eight adversarial datasets. |
Extracted BERT Model Leaks More Information than You Think! (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-trained language models are vulnerable to model extraction attacks . model extraction can cause severe privacy leakage even when victim models are facilitated with state-of-the-art defensive strategies. |
| Approach: | They propose to launch an attribute-inference attack against an extracted BERT model to prevent privacy leakage. |
| Outcome: | The proposed attack can cause severe privacy leakage even when victim models are facilitated with state-of-the-art defensive strategies. |
Here’s a Free Lunch: Sanitizing Backdoored Models with Model Merge (2024.findings-acl)
Copied to clipboard
| Challenge: | democratization of pre-trained language models brings significant security risks, including backdoor attacks. |
| Approach: | They propose to merge a backdoored model with other homogeneous models to remediate backdoor vulnerabilities. |
| Outcome: | The proposed model merging approach outperforms other models on classification tasks without additional resources or specific knowledge. |
Student Surpasses Teacher: Imitation Attack for Black-Box NLP APIs (2022.coling-1)
Copied to clipboard
| Challenge: | Existing MLaaS models are vulnerable to imitation attacks, but none of the stolen models can outperform the original black-box APIs. |
| Approach: | They conduct unsupervised domain adaptation and multi-victim ensemble to show attackers could surpass victims. |
| Outcome: | The proposed model outperforms the original black-box models on transferred domains. |
Dynamic Programming Encoding for Subword Segmentation in Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Empirical results on machine translation suggest that DPE is effective for segmenting output sentences. |
| Approach: | They propose a new algorithm for tokenizing sentences into subword units . they propose enabling exact log marginal likelihood estimation and exact MAP inference . |
| Outcome: | The proposed algorithm improves on machine translation datasets and on a large dataset. |
Self-Training Large Language Models for Tool-Use Without Demonstrations (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent work augmented LLMs with tools to mitigate factual inaccuracies and computational errors. |
| Approach: | They propose a method to synthesise tool-use traces using the LLM itself. |
| Outcome: | The proposed method improves performance on a long-tail knowledge task, but not on other datasets. |
Security Challenges in Natural Language Processing Models (2023.emnlp-tutorial)
Copied to clipboard
| Challenge: | Large-scale natural language processing models are vulnerable to security issues, such as backdoor attacks, private data leakage, and imitation attacks. |
| Approach: | They will dive into three emerging security issues in NLP research, i.e., backdoor attacks, private data leakage, and imitation attacks. |
| Outcome: | This tutorial will cover three emerging security issues in NLP research, i.e., backdoor attacks, private data leakage, and imitation attacks. |
Cut the Deadwood Out: Backdoor Purification via Guided Module Substitution (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Model NLP models are often trained on datasets from untrusted platforms, posing significant risks of data poisoning attacks. |
| Approach: | They propose a retraining-free method that selectively replaces modules in the victim model based on a trade-off signal between utility and backdoor. |
| Outcome: | The proposed method outperforms even the strongest defense baseline against challenging attacks like LWS. |
Are We Done with MMLU? (2025.naacl-long)
Copied to clipboard
Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong, Alessio Devoto, Alberto Carlo Maria Mancino, Rohit Saxena, Xuanli He, Yu Zhao, Xiaotang Du, Mohammad Reza Ghasemi Madani, Claire Barale, Robert McHardy, Joshua Harris, Jean Kaddour, Emile Van Krieken, Pasquale Minervini
| Challenge: | MMLU is widely adopted but its ground truth errors obscure the true capabilities of LLMs. |
| Approach: | They propose a framework for identifying dataset errors using a novel error annotation protocol and a subset of 5,700 manually re-annotated questions. |
| Outcome: | The proposed framework is based on 5,700 re-annotated questions from the MMLU benchmark. |
AfriMTE and AfriCOMET: Enhancing COMET to Embrace Under-resourced African Languages (2024.naacl-long)
Copied to clipboard
Jiayi Wang, David Adelani, Sweta Agrawal, Marek Masiak, Ricardo Rei, Eleftheria Briakou, Marine Carpuat, Xuanli He, Sofia Bourhim, Andiswa Bukula, Muhidin Mohamed, Temitayo Olatoye, Tosin Adewumi, Hamam Mokayed, Christine Mwase, Wangui Kimotho, Foutse Yuehgoh, Anuoluwapo Aremu, Jessica Ojo, Shamsuddeen Muhammad, Salomey Osei, Abdul-Hakeem Omotayo, Chiamaka Chukwuneke, Perez Ogayo, Oumaima Hourrane, Salma El Anigri, Lolwethu Ndolela, Thabiso Mangwana, Shafie Mohamed, Hassan Ayinde, Oluwabusayo Awoyomi, Lama Alkhaled, Sana Al-azzawi, Naome Etori, Millicent Ochieng, Clemencia Siro, Njoroge Kiragu, Eric Muchiri, Wangari Kimotho, Toadoum Sari Sakayo, Lyse Naomi Wamba, Daud Abolade, Simbiat Ajao, Iyanuoluwa Shode, Ricky Macharm, Ruqayya Iro, Saheed Abdullahi, Stephen Moore, Bernard Opoku, Zainab Akinjobi, Abeeb Afolabi, Nnaemeka Obiefuna, Onyekachi Ogbu, Sam Ochieng’, Verrah Otiende, Chinedu Mbonu, Yao Lu, Pontus Stenetorp
| Challenge: | Recent advances in machine translation (MT) have focused on scaling multilingual machine translation models and evaluation data to hundreds of languages, including multiple under-resourced languages. |
| Approach: | They propose to use n-gram matching metrics to measure progress in multilingual machine translation to 13 typologically diverse African languages to create high-quality human evaluation data with simplified MQM guidelines. |
| Outcome: | The proposed metrics have a higher correlation with human judgments than n-gram matching metrics such as BLEU and METEOR. |
Rethinking Round-Trip Translation for Machine Translation Evaluation (2023.findings-acl)
Copied to clipboard
| Challenge: | Automatic evaluation methods for translation often require model training and the availability of parallel corpora limits their applicability to low-resource settings. |
| Approach: | They revisit the statistical machine translation technique and use it to improve translation quality. |
| Outcome: | The proposed method improves translation quality estimation models and identifies adversarial competitors in shared tasks via cross-system verification. |
Generalised Unsupervised Domain Adaptation of Neural Machine Translation with Cross-Lingual Data Selection (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on unsupervised domain adaptation of neural machine translation assumes access to monolingual text in either the source or target language in the new domain. |
| Approach: | They propose a method to extract in-domain sentences from a large generic monolingual corpus from 'missing' text. |
| Outcome: | The proposed method outperforms baselines up to +1.5 BLEU score on five diverse domains in three language pairs and a real-world translation scenario. |
GRADA: Graph-based Reranking against Adversarial Documents Attack (2025.emnlp-main)
Copied to clipboard
Jingjie Zheng, Aryo Pradipta Gema, Giwon Hong, Xuanli He, Pasquale Minervini, Youcheng Sun, Qiongkai Xu
| Challenge: | Retrieval Augmented Generation (RAG) frameworks are susceptible to adversarial attacks that manipulate the retrieval process by introducing documents that are adversarially similar to the query. |
| Approach: | They propose a framework that integrates external retrieval modules into RAG frameworks to improve the factual accuracy of large language models. |
| Outcome: | The proposed framework reduces adversarial attacks by 80% while maintaining minimal loss in accuracy. |
Koala: An Index for Quantifying Overlaps with Pre-training Corpora (2023.emnlp-demo)
Copied to clipboard
| Challenge: | Recent studies have shown that large language models can be influenced by the frequency of overlap between pre-training corpora. |
| Approach: | They propose to search over large pre-training corpora using lossless compressed suffix arrays with highly efficient compression rate and search support. |
| Outcome: | Koala is a searchable index over large pre-training corpora using lossless compressed suffix arrays with highly efficient compression rate and search support. |
Scene Graph Modification Based on Natural Language Commands (2020.findings-emnlp)
Copied to clipboard
Xuanli He, Quan Hung Tran, Gholamreza Haffari, Walter Chang, Zhe Lin, Trung Bui, Franck Dernoncourt, Nhan Dam
| Challenge: | Numerous parsing methods have been developed for a single sentence, while a typical human-computer interaction session or conversation is not singleturn. |
| Approach: | They propose to modify an existing scene graph given a new user's command by using graph-based sparse transformer and cross attention information fusion to improve performance. |
| Outcome: | The proposed models outperform previous systems adapted from the machine translation and graph generation literature and contribute to the research community. |
Model Extraction and Adversarial Transferability, Your BERT is Vulnerable! (2021.naacl-main)
Copied to clipboard
| Challenge: | Pretrained language models are used for natural language processing (NLP) but when they are deployed as a service, they can suffer from different attacks . |
| Approach: | They propose two defence strategies to protect the target model from adversarial attacks . they show that model extraction and adversarially transferable attacks can be effective . |
| Outcome: | The extracted model can lead to highly transferable adversarial attacks against the target model. |
Foiling Training-Time Attacks on Neural Machine Translation Systems (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Neural machine translation systems are vulnerable to backdoor attacks . successful backdoors can cause slander, hate speech, phishing, etc. attacks can target very short trigger phrases, which can be challenging to detect even when included verbatim in poisoned instances. |
| Approach: | They propose a method that exploits asymmetry between source and target sentences to detect outlier tokens. |
| Outcome: | The proposed method reduces the success of attacks by up to 89.0% while not affecting predictive accuracy. |
Backdoor Attacks on Multilingual Machine Translation (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that multilingual machine translation systems are vulnerable to backdoor attacks through data poisoning. |
| Approach: | They propose to investigate the security of multilingual machine translation systems by exposing poisoned data into low-resource languages to cause malicious translations. |
| Outcome: | The proposed method achieves an average of 20% success rate in attacking high-resource languages. |
IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models (2025.naacl-long)
Copied to clipboard
David Ifeoluwa Adelani, Jessica Ojo, Israel Abebe Azime, Jian Yun Zhuang, Jesujoba Oluwadara Alabi, Xuanli He, Millicent Ochieng, Sara Hooker, Andiswa Bukula, En-Shiun Annie Lee, Chiamaka Ijeoma Chukwuneke, Happy Buzaaba, Blessing Kudzaishe Sibanda, Godson Koffi Kalipe, Jonathan Mukiibi, Salomon Kabongo Kabenamualu, Foutse Yuehgoh, Mmasibidi Setaka, Lolwethu Ndolela, Nkiruka Odu, Rooweither Mabuya, Salomey Osei, Shamsuddeen Hassan Muhammad, Sokhar Samb, Tadesse Kebede Guge, Tombekai Vangoni Sherman, Pontus Stenetorp
| Challenge: | Large language models (LLMs) are limited to a few high-resource languages . many low-resourced languages are evaluated only on basic text classification tasks . |
| Approach: | They propose to use IrokoBench to evaluate 17 low-resource African languages . they use human-translated benchmark datasets to evaluate zero-shot, few-shot and translate-test settings . |
| Outcome: | The proposed model performs well in English and French, but the highest performing model perform poorly in proprietary models. |
Generate, Annotate, and Learn: NLP with Synthetic Text (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing methods to generate unlabeled text are difficult to find. |
| Approach: | They propose a general framework called "generate, annotate, and learn" to take advantage of synthetic text within knowledge distillation, self-training, and few-shot learning applications. |
| Outcome: | The proposed framework achieves state-of-the-art knowledge distillation results for 6-layer transformers on the GLUE leaderboard. |
TUBA: Cross-Lingual Transferability of Backdoor Attacks in LLMs with Instruction Tuning (2025.findings-acl)
Copied to clipboard
Xuanli He, Jun Wang, Qiongkai Xu, Pasquale Minervini, Pontus Stenetorp, Benjamin I. P. Rubinstein, Trevor Cohn
| Challenge: | Despite the increasing support for multilingual capabilities, the impact of backdoor attacks on LLMs remains under-explored. |
| Approach: | They propose to use poisoned instructiontuning data to attack multilingual LLMs . their results show that more powerful models show increased susceptibility to transferable cross-lingual backdoor attacks . |
| Outcome: | The proposed attack is effective in models like BLOOM and GPT-4o with high success rates in more than 7 out of 12 languages. |