TrapDoc: Deceiving LLM Users by Injecting Imperceptible Phantom Tokens into Documents (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a growing number of LLMs have been used to provide reasoning, writing, text-editing capabilities. |
| Approach: | They propose a method to inject imperceptible phantom tokens into LLMs to deceive users . the technique generates outputs that appear plausible to users but are in fact incorrect . |
| Outcome: | a new method injects imperceptible tokens into documents to deceive users . the proposed framework is compared to baselines to show its effectiveness . |
Similar Papers
ContraDoc: Understanding Self-Contradictions in Documents with Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Detecting contradictions in texts is often regarded as determining relation between hypothesis and piece of premise. |
| Approach: | They propose a human-annotated dataset to study self-contradictions in long documents . they analyze the capabilities of four open-source and commercially available LLMs . |
| Outcome: | The proposed dataset outperforms open-source LLMs on document-level tasks but struggles with self-contradictions that require more nuance and context. |
Do LLMs Behave as Claimed? Investigating How LLMs Follow Their Own Claims using Counterfactual Questions (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation frameworks rely on curated datasets that, once public, may be accessed by newer LLMs. |
| Approach: | They propose a framework that generates counterfactual questions and answers from existing evaluation datasets and uses them to evaluate LLMs. |
| Outcome: | The proposed evaluation framework reduces the risk of data leakage by allowing the LLMs to respond to counterfactual questions and verify their claims. |
User Inference Attacks on Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | a large amount of data written by humans is used to train and fine-tune large language models. |
| Approach: | They propose to infer if a user's data was used to train an LLM by using example-level differential privacy. |
| Outcome: | The proposed attacks are easy to employ and only require black-box access to an LLM and a few samples from the user. |
Causal2Vec: Improving Decoder-only LLMs as Embedding Models through a Contextual Token (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods modify attention mechanism to be bidirectional, undermining LLMs’ ability to extract semantic information acquired during pre-training. |
| Approach: | They propose a general-purpose embedding model that pre-encodes input text into a single Contextual token and then prepends it to the LLM's input sequence. |
| Outcome: | The proposed model improves performance of decoder-only large language models without altering their architectures or introducing significant computational overhead. |
How to Make LLMs Forget: On Reversing In-Context Knowledge Edits (2025.naacl-long)
Copied to clipboard
| Challenge: | In-context knowledge editing (IKE) is an efficient and efficient knowledge editing method (Zheng et al., 2022b; Gangadhar and Stratos, 2024) it can be misused to manipulate responses opaquely, e.g., insert misinformation or offensive content. |
| Approach: | They propose to detect and reverse IKE-edits using only the top-10 output probabilities of the next token, even in a black-box setting. |
| Outcome: | The proposed method can be detected with high accuracy even in a black-box setting, achieving over 80% accuracy in recovering original, unedited outputs across multiple LLMs. |
Language Models can Subtly Deceive Without Lying: A Case Study on Strategic Phrasing in Legislation (2025.acl-long)
Copied to clipboard
Atharvan Dogra, Krishna Pillutla, Ameet Deshpande, Ananya B. Sai, John J Nay, Tanmay Rajpurohit, Ashwin Kalyan, Balaraman Ravindran
| Challenge: | blatant lying or unintentional hallucination are common in large language models. |
| Approach: | They build a testbed mimicking a legislative environment where a corporate lobbyist module is proposing amendments to bills that benefit a specific company while evading identification by strong LLM detectors. |
| Outcome: | The proposed model can be used to detect deception in legislative environments and to optimize its phrasing to avoid detection by strong detectors. |
Your Language Model Can Secretly Write Like Humans: Contrastive Paraphrase Attacks on LLM-Generated Text Detectors (2025.emnlp-main)
Copied to clipboard
Hao Fang, Jiawei Kong, Tianqu Zhuang, Yixiang Qiu, Kuofeng Gao, Bin Chen, Shu-Tao Xia, Yaowei Wang, Min Zhang
| Challenge: | Existing methods to detect large language models (LLMs) generated for plagiarism use paraphrases to rewrite them to evade detection. |
| Approach: | They propose a training-free method that effectively fools text detectors using off-the-shelf LLMs by rewriting them to evade detection. |
| Outcome: | The proposed method deceives text detectors using off-the-shelf LLMs by rewriting them to produce human-like sentences that are less discernible by detectors. |
Look Who’s Talking Now: Covert Channels From Biased LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | steganography encodes hidden messages into model-generated tokens . tradeoff between how much hidden information can be introduced and how much the model can be perturbed is important . |
| Approach: | They propose to use large language model-based steganography to encode hidden messages into model-generated tokens. |
| Outcome: | The proposed techniques are nearly optimal under a practical but difficult set of constraints . the proposed techniques ensure that only someone with the appropriate decoding key can access the hidden information . |
Boosting LLM Agents with Recursive Contemplation for Effective Deception Handling (2024.findings-acl)
Copied to clipboard
Shenzhi Wang, Chang Liu, Zilong Zheng, Siyuan Qi, Shuo Chen, Qisen Yang, Andrew Zhao, Chaofei Wang, Shiji Song, Gao Huang
| Challenge: | Recent advances in large language models (LLMs) have led to significant success in using LLMs as agents. |
| Approach: | They propose a cognitive framework that incorporates first-order and second-order perspective transitions into LLMs to enhance their ability to identify and counteract deceptive information. |
| Outcome: | The proposed framework enhances LLMs’ ability to identify and counteract deceptive information without extra fine-tuning and data. |
To Lie or Not to Lie? Investigating The Biased Spread of Global Lies by LLMs (2026.acl-long)
Copied to clipboard
Zohaib Khan, Mustafa Dogan, Ifeoma Okoh, Pouya Sadeghi, Siddhartha Shrestha, Sergius Justus Chesami Nyah, Mahmoud O. Mokhiamar, Michael J Ryan, Tarek Naous
| Challenge: | Misinformation is on the rise, and the strong writing capabilities of LLMs lower the barrier for malicious actors to produce and disseminate false information. |
| Approach: | They introduce a multilingual parallel dataset of 440 misinformation generation prompt templates and 6,867 entities, spanning 8 languages and 195 countries. |
| Outcome: | The proposed model reduces misinformation generation across languages and countries . it also reduces the risk of misinformation being spread across countries based on the model's performance . |