Papers by Thai Le
MULTITuDE: Large-Scale Multilingual Machine-Generated Text Detection Benchmark (2023.emnlp-main)
Copied to clipboard
Dominik Macko, Robert Moro, Adaku Uchendu, Jason Lucas, Michiharu Yamashita, Matúš Pikuliak, Ivan Srba, Thai Le, Dongwon Lee, Jakub Simko, Maria Bielikova
| Challenge: | MULTITuDE benchmarks lack authentic and machine-generated text in languages other than English . defining characteristic of new generation of LLMs is increased quality of text . |
| Approach: | They propose a benchmarking dataset for multilingual machine-generated text detection that compares detectors with authentic and machine-generated texts in 11 languages. |
| Outcome: | The proposed dataset compares detectors with zero-shot and fine-tuned detectors in 11 languages. |
“Are Your Explanations Reliable?” Investigating the Stability of LIME in Explaining Text Classifiers by Marrying XAI and Adversarial Attack (2023.emnlp-main)
Copied to clipboard
| Challenge: | Extensive experiments on text datasets demonstrate that XAIFooler significantly outperforms all baselines by large margins in its ability to manipulate LIME’s explanations with high semantic preservability. |
| Approach: | They propose to use LIME to establish a baseline and then propose an algorithm to perturb text inputs and manipulate explanations. |
| Outcome: | The proposed algorithm outperforms baselines on text datasets and achieves high semantic preservability. |
UPTON: Preventing Authorship Leakage from Public Text Release via Data Poisoning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent authorship attribution models can reveal the true authorship of unseen texts with high accuracies, with some cases up to 95% accuracy. |
| Approach: | They propose a solution that weakens authorship features in training samples and makes released texts unlearnable by exploiting black-box data poisoning methods. |
| Outcome: | The proposed model weakens authorship features in training samples and makes released texts unlearnable. |
A Curious Case of Searching for the Correlation between Training Data and Adversarial Robustness of Transformer Textual Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that fine-tuned textual transformer models are vulnerable to adversarial text perturbations. |
| Approach: | They extract 13 different features representing a wide range of input fine-tuning corpora properties and use them to predict adversarial robustness of the fine- tuned models. |
| Outcome: | The proposed framework can be used as an additional tool for robustness evaluation since it saves 30x-193x runtime compared to the traditional technique and can be easily used under adversarial training. |
HANSEN: Human and AI Spoken Text Benchmark for Authorship Analysis (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Authorship Analysis is an essential aspect of Natural Language Processing (NLP) for a long time. |
| Approach: | They propose to use 17 human speech datasets and 3 LLMs to create a benchmark for spoken texts. |
| Outcome: | The proposed benchmark encompasses 17 human datasets and AI-generated spoken texts created using 3 prominent LLMs: ChatGPT, PaLM2, and Vicuna13B. |
PlagBench: Exploring the Duality of Large Language Models in Plagiarism Generation and Detection (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent studies have raised concerns about the potential threats large language models pose to academic integrity and copyright protection. |
| Approach: | They propose a dataset of 46.5K synthetic text pairs that represent three major types of plagiarism: verbatim copying, paraphrasing, and summarization. |
| Outcome: | The proposed dataset shows that GPT-3.5 Turbo can produce high-quality paraphrases and summaries without significantly increasing text complexity compared to GPT-4 Turbo. |
PARASITE: Conditional System Prompt Poisoning to Hijack LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly deployed via third-party system prompts downloaded from public marketplaces. |
| Approach: | They propose a framework that optimizes system prompts to trigger LLMs to output compromised responses only for specific queries. |
| Outcome: | The proposed framework achieves up to 70% F1 reduction on targeted queries with minimal degradation to general capabilities. |
The Dangers of Indirect Prompt Injection Attacks on LLM-based Autonomous Web Navigation Agents: A Demonstration (2025.emnlp-demos)
Copied to clipboard
| Challenge: | Large Language Model (LLM)-integrated applications are becoming more popular to support, augment, and automate tasks. |
| Approach: | They propose to embed universal adversarial triggers in webpage HTML to hijack agents . they also use a browser-gym agent powered by Llama-3.1 to test their system . |
| Outcome: | The proposed system software is released under the MIT License . |
TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in generative language models have enabled machines to generate realistic texts. |
| Approach: | They propose a benchmark environment to test the 'Turing Test' problem for neural text generation methods. |
| Outcome: | The proposed benchmark environment is based on 200K human- or machine-generated samples across 20 labels Human, GPT-1, GTP-2_small, GTT-2_medium, GPG-2_large, GGT-2_PyTorch, GGP-3, GROVER_base, griover_large and GRover_mega. |
A Sweet Rabbit Hole by DARCY: Using Honeypots to Detect Universal Trigger’s Adversarial Attacks (2021.acl-long)
Copied to clipboard
| Challenge: | Existing adversarial examples can fool ML models by generating a fixed phrase that can drop the prediction accuracy of a textual neural network (NN) model to near zero on a target class. |
| Approach: | They propose a honeypot-based defense framework that greedily searches and injects multiple trapdoors into an NN model to “bait and catch” potential attacks. |
| Outcome: | The proposed algorithm detects attacks with 99% TPR and less than 2% FPR while maintaining prediction accuracy within 1% margin. |
Adapters Mixup: Mixing Parameter-Efficient Adapters to Enhance the Adversarial Robustness of Fine-tuned Pre-trained Text Classifiers (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that augmenting the training data of pre-trained language models with parametric fine-tuning methods can enhance their robustness under adversarial attacks. |
| Approach: | They propose an approach that fine-tunes PLMs with adapters and adversarial augmentation via mixup to leverage existing knowledge from a set of pre-known attacks. |
| Outcome: | The proposed approach achieves best trade-off between training efficiency and robustness under adversarial attacks compared to baselines on five downstream tasks across six varied black-box attacks and 2 PLMs. |
Authorship Attribution for Neural Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in deep learning have enabled the generation of realistic artifacts . however, the qualities of texts generated by these models are better, often confusing classifiers if they are not real. |
| Approach: | They propose to use neural network-based language models to generate realistic texts . they investigate the authorship attribution problem in three versions of a text . |
| Outcome: | The proposed models generate texts that are difficult to distinguish from human-written ones . the results show that most generators still generate texts significantly different from human ones compared to other models . |
Unraveling Interwoven Roles of Large Language Models in Authorship Privacy: Obfuscation, Mimicking, and Verification (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models have been driven by large-scale training corpora drawn from diverse sources such as websites, news articles, and books. |
| Approach: | They propose a framework for analyzing dynamic relationships among LLM-enabled AO, AM, and AV in the context of authorship privacy. |
| Outcome: | The proposed framework analyzes the dynamic relationships among LLM-enabled AO, AM, and AV in the context of authorship privacy. |
Perturbations in the Wild: Leveraging Human-Written Text Perturbations for Realistic Adversarial Attack and Defense (2022.findings-acl)
Copied to clipboard
| Challenge: | ANTHRO extracts over 600K human-written text perturbations and leverages them for realistic adversarial attacks. |
| Approach: | They propose an adversarial text manipulation algorithm that inductively extracts over 600K human-written text perturbations and leverages them for realistic adversarials. |
| Outcome: | The proposed algorithm outperforms the TextBugger baseline with an increase of 50% and 40% in terms of semantic preservation and stealthiness when evaluated by layperson and professional human workers. |
What You Read Isn’t What You Hear: Linguistic Sensitivity in Deepfake Speech Detection (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in text-to-speech technology have enabled highly realistic voice generation, fueling deepfake attacks. |
| Approach: | They propose a framework for transcript-to-audio perturbation anti-spoofing that incorporates linguistic variation into detectors to investigate spoof detection. |
| Outcome: | The proposed framework can bypass commercial detectors by incorporating linguistic variation into the design of anti-spoofing systems. |
SHIELD: Defending Textual Neural Networks against Multiple Black-Box Adversarial Attacks with Stochastic Multi-Expert Patcher (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to defend textual neural network models against adversarial attacks often require retraining and retrain . e.g., BERT, RoBERTa require great time and computation resources. |
| Approach: | They propose an algorithm that modifies and re-trains only the last layer of a textual NN and transforms it into a stochastic weighted ensemble of multi-expert prediction heads. |
| Outcome: | The proposed algorithm outperforms existing models against black-box attacks by 15%–70% . the proposed algorithm is based on a novel algorithm from software engineering . |
A Ship of Theseus: Curious Cases of Paraphrasing in LLM-Generated Texts (2024.acl-long)
Copied to clipboard
Nafis Irtiza Tripto, Saranya Venkatraman, Dominik Macko, Robert Moro, Ivan Srba, Adaku Uchendu, Thai Le, Dongwon Lee
| Challenge: | Using a computational approach, we discover that diminishing performance in text classification models is closely associated with the extent of deviation from the original author’s style. |
| Approach: | They propose to use large language models to determine whether a text retains original authorship when it undergoes numerous paraphrasing iterations. |
| Outcome: | The results suggest that authorship should be task-dependent . |
Catch Me If You GPT: Tutorial on Deepfake Texts (2024.naacl-tutorials)
Copied to clipboard
| Challenge: | In recent years, natural language generation (NLG) techniques have advanced, but pose new security risks . this tutorial will be 3 hours long with a mix of lecture and hands-on examples for interactive audience participation. |
| Approach: | They present a tutorial on the security of natural language generation (NLG) they review the latest literature on the detection and obfuscation of deepfake text authorships . |
| Outcome: | This tutorial reviews the latest literature on the detection and obfuscation of deepfake text authorships. |
Harry Potter is Still Here! Probing Knowledge Leakage in Targeted Unlearned Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing frameworks for unlearning models can leak idiosyncratic information under adversarial conditions. |
| Approach: | They propose a framework that probes for undesired knowledge retention in unlearned LLMs . LURK automatically generates adversarial suffixes designed to elicit residual knowledge . |
| Outcome: | The proposed framework detects undesired knowledge retention in unlearned models . it generates adversarial suffix prompts to elicit residual knowledge about the Harry Potter domain . |
Generalizability of Mixture of Domain-Specific Adapters from the Lens of Signed Weight Directions and its Application to Effective Model Pruning (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning Pretrained Language Models (PLMs) focus on training multiple adapters and continuously adding more adapters for incoming new tasks, making learning among the tasks unequal. |
| Approach: | They propose to mix only a select sub-set of adapters during inference via model weight averaging to optimize performance on unseen, in-domain examples. |
| Outcome: | The proposed methods can integrate a single specialized knowledge into existing Pre-Trained Language Models (PLMs) but only a select sub-set of adapters during inference on unseen, in-domain examples. |