Watermark under Fire: A Robustness Evaluation of LLM Watermarking (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Various watermarking methods have been proposed to identify LLM-generated texts . lack of unified evaluation platforms has left many critical questions unanswered . |
| Approach: | They systematize existing LLM watermarkers and watermark removal attacks and develop a unified platform that integrates them. |
| Outcome: | The proposed systematizes existing LLM watermarkers and watermark removal attacks, mapping out their design spaces. |
Similar Papers
WaterBench: Towards Holistic Evaluation of Watermarks for Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Recent studies have developed watermarking algorithms which restrict the generation process to leave an invisible trace for watermark detection. |
| Approach: | They propose a benchmarking procedure that compares different methods to ensure consistent watermarking strength and jointly evaluates their generation and detection performance. |
| Outcome: | The proposed benchmark compares 4 open-source watermarks on 2 LLMs under 2 watermarking strengths and observes the common struggles for current methods on maintaining the generation quality. |
Revisiting the Robustness of Watermarking to Paraphrasing Attacks (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent watermarking techniques slightly modify the output probabilities of LMs to embed a signal in the generated output that can later be detected. |
| Approach: | They propose to use a model to produce a watermarking signal that is invariant to semantically-similar inputs to undo the effects of watermarks. |
| Outcome: | The proposed method undoes the effects of watermarking and dramatically improves the effectiveness of paraphrasing attacks with limited access to model generations. |
MarkLLM: An Open-Source Toolkit for LLM Watermarking (2024.emnlp-demo)
Copied to clipboard
Leyi Pan, Aiwei Liu, Zhiwei He, Zitian Gao, Xuandong Zhao, Yijian Lu, Binglin Zhou, Shuliang Liu, Xuming Hu, Lijie Wen, Irwin King, Philip Yu
| Challenge: | Large Language Models (LLMs) embed imperceptible yet algorithmically detectable signals in outputs to identify LLM-generated text. |
| Approach: | They propose to develop an open-source toolkit for LLM watermarking that embeds imperceptible yet algorithmically detectable signals in model outputs to identify LLM-generated text. |
| Outcome: | MarkLLM provides a unified framework for implementing LLM watermarking algorithms, while providing user-friendly interfaces to ensure ease of access. |
WaterJudge: Quality-Detection Trade-off when Watermarking Large Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent work has shown that small, context-dependent shifts in word distributions can be used to apply and detect watermarks, but little work has analyzed the impact of these perturbations on the quality of generated texts. |
| Approach: | They propose a framework that allows for analysis of the impact of watermark settings on the quality of generated texts. |
| Outcome: | The proposed framework provides easy visualization of the quality-detection trade-off of watermark settings. |
A Reinforcement Learning Framework for Robust and Secure LLM Watermarking (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing watermarking algorithms rely on heuristic green/red token lists . however, these lists are inconsistent and can be compromised . |
| Approach: | They propose a framework for robust and secure LLM watermarking using reinforcement learning. |
| Outcome: | The proposed method achieves state-of-the-art trade-off across all criteria with notable improvements in resistance to spoofing attacks without degrading other criteria. |
PostMark: A Robust Blackbox Watermark for Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to detect LLM-generated text require access to the underlying LLM’s logits, which LLM providers are loath to share due to fears of model distillation. |
| Approach: | They develop a post-hoc watermarking procedure that inserts an input-dependent set of words into the text after the decoding process has completed. |
| Outcome: | The proposed method is more robust to paraphrasing attacks than existing methods. |
Subtle Signatures, Strong Shields: Advancing Robust and Imperceptible Watermarking in Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have led to an increase in AI-generated text on the Internet, presenting a crucial challenge to differentiate AI-created content from human-written text. |
| Approach: | They propose a novel approach to embed watermarks into LLMs that leverages token prior probabilities to improve detectability and maintain watermark imperceptibility. |
| Outcome: | The proposed method improves detectability and imperceptibility of watermarks by partitioning tokens into two distinct groups based on prior probabilities and employing tailored strategies for each group. |
WaterPool: A Language Model Watermark Mitigating Trade-Offs among Imperceptibility, Efficacy and Robustness (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to trace the usage of large language models often face trade-offs between imperceptibility and robustness. |
| Approach: | They propose a key-centered scheme to unify existing methods by decomposing a watermark into two components: a 'key module' and a "mark module". |
| Outcome: | The proposed method can be integrated with existing methods and achieve near-optimal imperceptibility and detection efficacy. |
Lost in Overlap: Exploring Logit-based Watermark Collision in LLMs (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing watermarking methods embed imperceptible identifiers into text to address copyright concerns. |
| Approach: | They propose a new philosophy for watermark attacks that addresses watermark collision . they demonstrate that collision poses a threat to all logit-based watermark algorithms . |
| Outcome: | The proposed method improves watermark collision performance on top of other methods. |
From Intentions to Techniques: A Comprehensive Taxonomy and Challenges in Text Watermarking for Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are rapidly growing and allowing textual content to be protected against unauthorized use. |
| Approach: | They present a unified overview of different perspectives behind designing watermarking techniques through a comprehensive survey of the research literature. |
| Outcome: | The proposed methods are based on the evaluation datasets used and watermarking addition and removal methods to construct a taxonomy. |