On the Risk of Evidence Pollution for Malicious Social Text Detection in the Era of LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Evidence-enhanced detectors are able to detect malicious social text, but they are prone to evidence pollution. |
| Approach: | They propose three defense strategies to mitigate evidence pollution by large language models by machine-generated text detection and a mixture of experts. |
| Outcome: | The proposed defense strategies could mitigate evidence pollution, but they faced limitations for practical employment. |
Similar Papers
On the Risk of Misinformation Pollution with Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study demonstrates that large language models can be misused for generating credible-sounding misinformation . however, the ability to produce credible text raises concerns regarding their potential misuse . |
| Approach: | They propose three defense strategies to mitigate misinformation generated by Large Language Models . they propose a threat model and simulate potential misuse scenarios . |
| Outcome: | The proposed defense strategies have shown promising results, albeit with costs. |
What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detection (2024.acl-long)
Copied to clipboard
| Challenge: | Social media bot detection has always been an arms race between advancements in machine learning and adversarial bot strategies to evade detection. |
| Approach: | They propose a mixture-of-heterogeneous-experts framework to divide and conquer diverse user information modalities and propose LLM-guided manipulation of user textual and structured information to evade detection. |
| Outcome: | The proposed framework outperforms state-of-the-art baselines on 1,000 annotated examples while bringing down existing detectors by 29.6% and harming calibration and reliability of bot detection systems. |
How Susceptible are Large Language Models to Ideological Manipulation? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have the potential to exert substantial influence on public perceptions and interactions with information. |
| Approach: | They examine how LLMs can learn and generalize ideological biases from their instruction-tuning data. |
| Outcome: | The LLMs show a startling ability to absorb ideology from one topic and generalize it to even unrelated ones. |
SHIELD: Evaluation and Defense Strategies for Copyright Compliance in LLM Text Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have transformed machine learning but have raised significant legal concerns due to their potential to produce text that infringes on copyrights. |
| Approach: | They propose a lightweight, real-time defense mechanism to prevent the generation of copyrighted text by evaluating methods and testing attack strategies. |
| Outcome: | The proposed defense significantly reduces the volume of copyrighted text generated by LLMs by effectively refusing malicious requests. |
Data to Defense: The Role of Curation in Aligning Large Language Models Against Safety Compromise (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have identified a vulnerability in large language models (LLMs) during customization. |
| Approach: | They propose an adaptive data curation approach that allows any text to be curated to enhance its effectiveness in counteracting harmful samples during customization. |
| Outcome: | The proposed approach reduces compromising effects and generates 100% safe responses. |
Evaluation of LLM Vulnerabilities to Being Misused for Personalized Disinformation Generation (2025.acl-long)
Copied to clipboard
Aneta Zugecova, Dominik Macko, Ivan Srba, Robert Moro, Jakub Kopál, Katarína Marcinčinová, Matúš Mesarčík
| Challenge: | Recent large language models generate disinformation news articles following predefined narratives . personalization and disinformation abilities of LLMs have not been studied . |
| Approach: | They evaluate the personalization and disinformation abilities of large language models . they find personalization reduces the safety-filter activations, thus effectively functioning as a jailbreak . |
| Outcome: | The proposed model generates disinformation news articles in english with the lowest quality of personalization. |
Stumbling Blocks: Stress Testing the Robustness of Machine-Generated Text Detectors Under Attacks (2024.acl-long)
Copied to clipboard
| Challenge: | Existing studies on this topic focus on the robustness of specific detectors or particular attack methods. |
| Approach: | They stress test the detectors’ robustness to malicious attacks under realistic scenarios using LLMs and metric-based detectors. |
| Outcome: | The proposed methods are based on a set of LLM-based models and their performance is compared under different budget levels. |
Does Data Contamination Detection Work (Well) for LLMs? A Survey and Evaluation on Detection Assumptions (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large language models have demonstrated great performance across various benchmarks, but data contamination is a concern in their evaluation. |
| Approach: | They analyze 50 papers on data contamination detection and test three of them as case studies to identify the possibility of data contamination. |
| Outcome: | The proposed methods can detect membership inference attacks on instance-level data, and can perform similar to random guessing on LLM pretraining datasets. |
Towards Data Contamination Detection for Modern Large Language Models: Limitations, Inconsistencies, and Oracle Challenges (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for detecting data contamination in large language models have limitations and limitations . data contamination occurs when test or evaluation data is exposed to the model during its training phases . |
| Approach: | They evaluate five different methods for detecting data contamination in large language models . they find that current methods have non-trivial limitations in their assumptions and practical applications . |
| Outcome: | The proposed methods have non-trivial limitations and difficulties in detecting contamination . the authors highlight the complexity of contamination detection in advanced LLMs . |
A Survey on Detection of LLMs-Generated Content (2024.findings-emnlp)
Copied to clipboard
Xianjun Yang, Liangming Pan, Xuandong Zhao, Haifeng Chen, Linda Petzold, William Yang Wang, Wei Cheng
| Challenge: | Recent advances in large language models have led to an increase in synthetic content generation . the ability to detect LLMs-generated content has become of paramount importance . |
| Approach: | They propose to provide a detailed overview of existing detection strategies and benchmarks, scrutinizing their differences and advocating for more adaptable and robust models to enhance detection accuracy. |
| Outcome: | The proposed model will be able to detect human-written content in real time. |