HQP: A Human-Annotated Dataset for Detecting Online Propaganda (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets for detecting online propaganda use weak labels that can be noisy and incorrect. |
| Approach: | They propose a dataset for detecting online propaganda with high-quality labels . they show that state-of-the-art language models fail in detecting propaganda when trained with weak labels compared to prompt-based learning . |
| Outcome: | The proposed dataset is the first large-scale dataset for detecting online propaganda that was created through human annotation. |
Similar Papers
KinyaProp: Fine-Grained Propaganda Annotation in Kinyarwanda (2026.acl-long)
Copied to clipboard
| Challenge: | Propaganda is a widely used approach for shaping public opinion and disseminating misinformation in news media. |
| Approach: | They propose a fine-grained propaganda dataset for Kinyarwanda . they find that current LLMs are not reliable annotators in low resource settings . |
| Outcome: | The proposed dataset shows that current LLMs perform poorly in low resource settings . the dataset shows they perform poorly on discourse-level techniques . |
Fine-tuning with Hierarchical Prompting for Robust Propaganda Classification Across Annotation Schemas (2026.findings-acl)
Copied to clipboard
Lukas Stähelin, Veronika Solopova, Max Upravitelev, David Kaplan, Premtim Sahitaj, Ariana Sahitaj, Charlott Jakob, Sebastian Möller, Vera Schmitt
| Challenge: | Propaganda detection in social media is challenging due to noisy, short texts and low annotation agreements. |
| Approach: | They propose a new intent-focused taxonomy of propaganda techniques and compare it against an established, higher-agreement schema. |
| Outcome: | The proposed taxonomy outperforms existing models and reveals methodological differences hidden in base models. |
Large Language Models for Propaganda Span Annotation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Using propagandistic techniques to manipulate online audiences is increasing in recent years. |
| Approach: | They investigate whether Large Language Models (LLMs) such as GPT-4 can extract propagandistic spans and the potential of employing them to collect more cost-effective annotations. |
| Outcome: | The proposed model provides labels that have higher agreement with expert annotators and lead to specialized models that achieve state-of-the-art over an unseen Arabic testing set. |
Can GPT-4 Identify Propaganda? Annotation and Detection of Propaganda Spans in News Articles (2024.lrec-main)
Copied to clipboard
| Challenge: | Using large language models (LLMs) to detect propaganda from text is a challenge for the development of sophisticated models. |
| Approach: | They propose to use a large propaganda dataset to identify propagandistic content in text, visual, or multimodal languages to improve their models. |
| Outcome: | The proposed model performs better on a large propaganda dataset than the existing models on skewed datasets. |
On Sentence Representations for Propaganda Detection: From Handcrafted Features to Word Embeddings (D19-50)
Copied to clipboard
| Challenge: | The rise of fake and hyperpartisan news on social media and online news outlets calls for improved automatic detection of propaganda in texts. |
| Approach: | They propose to use handcrafted features and learn dense semantic representations to detect propaganda in sentence-level and with random undersampling of the majority class (non-propaganda) |
| Outcome: | The proposed system achieves a ranking of 10 among 25 participants, with 59.5 F1-score. |
Annotating the Annotators: Analysis, Insights and Modelling from an Annotation Campaign on Persuasion Techniques Detection (2025.findings-acl)
Copied to clipboard
Davide Bassi, Dimitar Iliyanov Dimitrov, Bernardo D’Auria, Firoj Alam, Maram Hasanain, Christian Moro, Luisa Orrù, Gian Piero Turchi, Preslav Nakov, Giovanni Da San Martino
| Challenge: | Existing annotation campaigns based on heuristic guidelines have not been thoroughly discussed. |
| Approach: | They propose a probabilistic model for optimizing intervention scheduling to reduce the cost of an expert oversight in annotation tasks. |
| Outcome: | The proposed model advocates for an expert oversight in annotation tasks and periodic quality audits to reduce costs. |
The Promises and Pitfalls of LLM Annotations in Dataset Labeling: a Case Study on Media Bias Detection (2025.findings-naacl)
Copied to clipboard
Tomáš Horych, Christoph Mandl, Terry Ruas, Andre Greiner-Petter, Bela Gipp, Akiko Aizawa, Timo Spinde
| Challenge: | Recent research suggests using Large Language Models (LLMs) to automate the annotation process, reducing these costs while maintaining data quality. |
| Approach: | They propose to use Large Language Models to automate annotation process and train classifiers on large datasets. |
| Outcome: | The proposed model outperforms all of the annotator LLMs on two media bias benchmark datasets (BABE and BASIL) while maintaining data quality. |
JUSTDeep at NLP4IF 2019 Task 1: Propaganda Detection using Ensemble Deep Learning Models (D19-50)
Copied to clipboard
| Challenge: | Detecting fake news is not well established yet, but it can be classified under several labels: false, biased, or framed to mislead the readers. |
| Approach: | They propose a deep learning model using BiLSTM, XGBoost, and BERT to detect propaganda using a corpus from a challenge. |
| Outcome: | The proposed model outperforms the baseline model on a dataset from the challenge NLP4IF 2019 . |
Beyond Dataset Creation: Critical View of Annotation Variation and Bias Probing of a Dataset for Online Radical Content Detection (2025.coling-main)
Copied to clipboard
| Challenge: | Existing datasets and models fail to address the complexities of multilingual data, authors say . detection of radical content on online platforms has become an increasingly pressing concern . |
| Approach: | They propose a publicly available multilingual dataset annotated with radicalization levels, calls for action, and named entities in English, French, and Arabic. |
| Outcome: | The proposed dataset is annotated with radicalization levels, calls for action, and named entities in English, French, and Arabic. |
Detecting Label Errors by Using Pre-Trained Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for label error detection focus on label errors in training data. |
| Approach: | They propose a method for introducing realistic, human-originated label noise into existing crowdsourced datasets such as SNLI and TweetNLP. |
| Outcome: | The proposed method outperforms existing methods for detecting label errors in natural language datasets. |