Papers by Preslav Nakov
Exploring the Limitations of Detecting Machine-Generated Text (2025.coling-main)
Copied to clipboard
| Challenge: | Recent advances in the quality of the generation of text by large language models have spurred research into identifying machine-generated text. |
| Approach: | They audit classification performance for detecting machine-generated text by evaluating on texts with varying writing styles. |
| Outcome: | The proposed methods are highly sensitive to stylistic changes and complexity, and in some cases degrade entirely to random classifiers. |
Predicting the Topical Stance and Political Leaning of Media using Tweets (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for determining stances of media outlets and influential people are expensive. |
| Approach: | They propose a method that uses unsupervised learning to ascertain the stance of Twitter users with respect to a polarizing topic by leveraging their retweet behavior. |
| Outcome: | The proposed method achieves 82.6% accuracy compared to gold labels from the Media Bias/Fact Check website . |
TOP-Training: Target-Oriented Pretraining for Medical Extractive Question Answering (2025.coling-main)
Copied to clipboard
| Challenge: | e-health records underscore the growing significance of information extraction (IE) from these datasets. |
| Approach: | They propose a target-oriented pre-training paradigm for extractive question-answering in the medical domain . TOP-Training moves one step further than popular domain-oriented fine-tuning . |
| Outcome: | The proposed method improves on the Medical-EQA benchmarks. |
\mathsf{Con Instruction}: Universal Jailbreaking of Multimodal Large Language Models via Non-Textual Modalities (2025.acl-long)
Copied to clipboard
| Challenge: | Existing attacks communicate instruction through text, accompanied by a toxic image or audio . a novel gray-box attack method generates adversarial images or audio to convey harmful instructions to MLLMs . |
| Approach: | They propose a gray-box attack method that generates adversarial images or audio to convey specific harmful instructions to MLLMs by following non-textual instruction. |
| Outcome: | The proposed method achieves highest success rates on visual and audio-language models . larger models are more susceptible toCon Instruction, compared to their underlying models - the results will be released . |
A Survey on Multimodal Disinformation Detection (2022.coling-1)
Copied to clipboard
Firoj Alam, Stefano Cresci, Tanmoy Chakraborty, Fabrizio Silvestri, Dimiter Dimitrov, Giovanni Da San Martino, Shaden Shaar, Hamed Firooz, Preslav Nakov
| Challenge: | Recent years have witnessed the proliferation of offensive content online such as fake news, propaganda, misinformation, and disinformation. |
| Approach: | They propose to tackle online multimodal offensive content using different modalities and combinations thereof. |
| Outcome: | The proposed approach combines factuality and harmfulness in a framework that can be used for multiple modalities and combinations of modality. |
Cross-Cultural Transfer of Commonsense Reasoning in LLMs: Evidence from the Arab World (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study examined the potential for cross-cultural transfer of commonsense reasoning . merely 12 culture-specific examples from one country can improve performance in others by 10% on average . |
| Approach: | They evaluate cross-cultural transfer of commonsense reasoning within the arab world . they use in-context learning and demonstration-based reinforcement to evaluate alignment methods . |
| Outcome: | The proposed model can improve performance in cultures with cultural similarities in the Arab world by 10% on average. |
Compressing Large-Scale Transformer-Based Models: A Case Study on BERT (2021.tacl-1)
Copied to clipboard
Prakhar Ganesh, Yao Chen, Xin Lou, Mohammad Ali Khan, Yin Yang, Hassan Sajjad, Preslav Nakov, Deming Chen, Marianne Winslett
| Challenge: | Popular pre-trained Transformers have improved performance for various NLP tasks by sizable margins, but are too resource-hungry and computation-intensive to suit low-capacity devices or applications with strict latency requirements. |
| Approach: | They present a literature review of the compression of Transformers, focusing on the popular BERT model, which has attracted considerable research attention. |
| Outcome: | The proposed models improve Sentiment analysis, paraphrase detection, machine reading comprehension, question answering, text summarization, and other tasks by sizable margins. |
Predicting the Type and Target of Offensive Posts in Social Media (N19-1)
Copied to clipboard
| Challenge: | Prior work focused on detecting specific types of offensive content, such as hate speech, cyberbullying, or cyber-aggression. |
| Approach: | They propose to use a dataset to identify offensive content in social media . they compare the performance of different machine learning models to OLID . |
| Outcome: | The proposed dataset contains tweets annotated for offensive content using a fine-grained three-layer annotation scheme. |
Adapting Fake News Detection to the Era of Large Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | a gap exists in understanding the interplay between machine-paraphrased real news, machine-generated fake news, and human-written real news . false information is easier to generate but harder to detect due to the bias of detectors against machine-generated texts . |
| Approach: | They propose a strategy to adapt fake news detectors to the era of large language models and AI-driven content creation . |
| Outcome: | The proposed detectors perform well on human-written articles but not vice versa . the proposed detector should be trained on datasets with lower machine-generated news ratio than the test set . |
Droid: A Resource Suite for AI-Generated Code Detection (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing detectors fail to generalise to diverse coding domains and programming languages outside of their narrow training data. |
| Approach: | They propose to use DroidCollection to train machine-generated code detectors that can be trained on a multi-task objective. |
| Outcome: | The proposed detectors fail to generalise to diverse coding domains and programming languages outside of their narrow training data. |
Target-Based Offensive Language Identification (2023.acl-short)
Copied to clipboard
Marcos Zampieri, Skye Morgan, Kai North, Tharindu Ranasinghe, Austin Simmmons, Paridhi Khandelwal, Sara Rosenthal, Preslav Nakov
| Challenge: | Popular social media annotation taxonomies focus on the post level and token-level annotations are not available. |
| Approach: | They propose a new dataset for Target-based Offensive language identification that uses post-level and token-level annotations to identify offensive language on Twitter. |
| Outcome: | The proposed taxonomy can be used to annotate offensive language on English Twitter posts. |
Beyond "Not Novel Enough": Enriching Scholarly Critique with LLM-Assisted Feedback (2026.eacl-long)
Copied to clipboard
| Challenge: | Novelty assessment is a central yet understudied aspect of peer review . manuscript submissions double roughly every 15 years, and individual reviewers now complete an average of 14 reviews per year. |
| Approach: | They propose a structured approach for automated novelty evaluation that models expert reviewer behavior through three stages: content extraction, retrieval and synthesis of related work, and structured comparison for evidence-based assessment. |
| Outcome: | The proposed approach outperforms existing LLM-based baselines on 182 ICLR 2025 submissions with human-annotated reviewer novelty assessments. |
M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection (2024.eacl-long)
Copied to clipboard
Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Chenxi Whitehouse, Osama Mohammed Afzal, Tarek Mahmoud, Toru Sasaki, Thomas Arnold, Alham Aji, Nizar Habash, Iryna Gurevych, Preslav Nakov
| Challenge: | Large language models generate fluent responses to user queries, but they are also susceptible to misuse in journalism, education, and academia. |
| Approach: | They propose a large-scale benchmark for machine-generated text detection that is a multi-generator, multi-domain, and multi-lingual corpus. |
| Outcome: | The proposed system can detect machine-generated text and pinpoint misuse . the proposed system is based on a large-scale benchmark dataset . |
We Can Detect Your Bias: Predicting the Political Ideology of News Articles (2020.emnlp-main)
Copied to clipboard
| Challenge: | a new study examines the role of media in predicting political ideology or bias in news articles . systematic exposure to bias in the news can foster intolerance and ideological segregation . |
| Approach: | They propose an adversarial media adaptation and a specially adapted triplet loss for predicting political ideology in news articles. |
| Outcome: | The proposed model improves over state-of-the-art models in this challenging setup. |
The Battlefront of Combating Misinformation and Coping with Media Bias (2022.aacl-tutorials)
Copied to clipboard
| Challenge: | a growing number of misinformation and misinformation is affecting our daily lives . a tutorial aims to address the challenges of detecting fake news and media bias . |
| Approach: | They provide an overview of the frontier in fighting misinformation . they propose to develop a robust fake news detection system to combat misinformation. |
| Outcome: | This tutorial examines the frontiers of fake news detection and media bias detection . it focuses on how to fact-check information pieces and uncover bias and agenda of news sources . |
Vector-Vector-Matrix Architecture: A Novel Hardware-Aware Framework for Low-Latency Inference in NLP Applications (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to improve accuracy of neural networks are slow due to computational complexity. |
| Approach: | They propose a vector-vector-matrix architecture which greatly reduces latency at inference time for NLP applications by a factor of four. |
| Outcome: | The proposed framework reduces the latency of sequence-to-sequence and Transformer models used for NMT by a factor of four. |
Adversarial Domain Adaptation for Duplicate Question Detection (D18-1)
Copied to clipboard
| Challenge: | Recent years have seen the rise of community question answering forums . duplicate questions easily become ubiquitous as users often ask the same question, possibly in a slightly different formulation, making it difficult to find the best (or one correct) answer. |
| Approach: | They propose to use domain adaptation to detect duplicate questions in forums . they find that domain adaptation improves performance over multiple pairs of domains . |
| Outcome: | The proposed approach improves 5.6% over the best baseline across multiple pairs of domains. |
Fact-Checking Complex Claims with Program-Guided Reasoning (2023.acl-long)
Copied to clipboard
| Challenge: | Fact-checking real-world claims often requires collecting multiple pieces of evidence and complex multi-step reasoning. |
| Approach: | They propose a novel fact-checking model that decomposes complex claims into simpler sub-tasks that can be solved using a shared library of specialized functions. |
| Outcome: | The proposed model outperforms seven baselines on two fact-checking datasets and has explicit output programs that benefit human debugging. |
COMMUNITYNOTES: A Dataset for Exploring the Helpfulness of Fact-Checking Explanations (2026.findings-eacl)
Copied to clipboard
| Challenge: | X, Meta, and TikTok are experimenting with community-based factchecking . community-driven verification is a way to provide explanatory notes that clarify why a post might be misleading . |
| Approach: | They propose a framework that optimizes the helpfulness of explanatory notes and the reason for this by automatically optimizing the prompt definitions. |
| Outcome: | The proposed framework improves helpfulness and reason prediction on 104k posts with user-provided notes and helpfulness labels. |
Qorǵau: Evaluating Safety in Kazakh-Russian Bilingual Contexts (2025.findings-acl)
Copied to clipboard
Maiya Goloburda, Nurkhan Laiyk, Diana Turmakhan, Yuxia Wang, Mukhammed Togmanov, Jonibek Mansurov, Askhat Sametov, Nurdaulet Mukhituly, Minghan Wang, Daniil Orel, Zain Muhammad Mujahid, Fajri Koto, Timothy Baldwin, Preslav Nakov
| Challenge: | Large language models (LLMs) have the potential to generate harmful content, posing risks to users. |
| Approach: | They propose a dataset specifically designed for safety evaluation in Kazakh and Russian . they use a bilingual context in Kazakhstan where both Kazakh (a low-resource language) and Russian (a high-resourced language) |
| Outcome: | The proposed dataset is designed for safety evaluation in Kazakh and Russian . it shows that both multilingual and language-specific LLMs perform better than others . |
A Chinese Dataset for Evaluating the Safeguards in Large Language Models (2024.findings-acl)
Copied to clipboard
Yuxia Wang, Zenan Zhai, Haonan Li, Xudong Han, Shom Lin, Zhenxuan Zhang, Angela Zhao, Preslav Nakov, Timothy Baldwin
| Challenge: | a recent study has shown that large language models can produce harmful responses, exposing users to unexpected risks. |
| Approach: | They propose a dataset for the safety evaluation of Chinese LLMs in Mandarin Chinese . they extend the dataset to better identify false negative and false positive examples . |
| Outcome: | The proposed dataset is for the safety evaluation of Chinese LLMs, and is based on a Chinese dataset. |
Missci: Reconstructing Fallacies in Misrepresented Science (2024.acl-long)
Copied to clipboard
| Challenge: | False or misleading narratives spread rapidly on social networks, posing challenges for non-experts in discerning credible information. |
| Approach: | They propose a model for fallacious reasoning that focuses on implicit fallacies between relevant content and the inaccurate claim and requires models to verbalize the fallacious thinking in addition to classifying it. |
| Outcome: | The proposed model focuses on implicit fallacies between relevant content and the inaccurate claim and requires models to verbalize the fallacious reasoning in addition to classifying it. |
From Chaos to Clarity: Claim Normalization to Empower Fact-Checking (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Social media posts are noisy and pervasive, resulting in difficult to identify precise and prominent claims that require verification. |
| Approach: | They propose a task called Claim Normalization that decomposes complex and noisy social media posts into more straightforward and understandable forms, termed normalized claims. |
| Outcome: | The proposed model outperforms baselines across evaluation measures and errors. |
FAID: Fine-grained AI-generated Text Detection using Multi-task Auxiliary and Multi-level Contrastive Learning (2026.eacl-long)
Copied to clipboard
Minh Ngoc Ta, Dong Cao Van, Duc-Anh Hoang, Minh Le-Anh, Truong Nguyen, My Anh Tran Nguyen, Yuxia Wang, Preslav Nakov, Dinh Viet Sang
| Challenge: | Existing binary detection frameworks for human-written, LLM-generated and human-LLM collaborative texts are challenging . a recent study focused on binary detection, i.e., human vs. LLM, or on fine-grained detection limited to English. |
| Approach: | They propose a fine-grained detection framework to classify text into three categories . they use multilingual datasets and a multi-domain, multi-generator dataset . |
| Outcome: | The proposed framework outperforms baselines on unseen domains and new LLMs. |
Fact-Checking Meets Fauxtography: Verifying Claims About Images (D19-1)
Copied to clipboard
| Challenge: | Recent explosion of false claims in social media has led to manual fact-checking initiatives . however, existing methods are inadequate to deal with the growing number of false content claims. |
| Approach: | They propose to model claims about images using a new dataset to examine the relationship between the image and the claim. |
| Outcome: | The proposed method improves on the baseline and will enable future research on fact-checking claims about images. |
ThinkBooster: A Unified Framework for Seamless Test-Time Scaling of LLM Reasoning (2026.acl-demo)
Copied to clipboard
Vladislav Smirnov, Quang-Chieu Nguyen, Sergey Senichev, Minh Ngoc Ta, Ekaterina Fadeeva, Artem Vazhentsev, Daria Galimzianova, Nikolai Rozanov, Viktor Mazanov, Jingwei Ni, Tianyi Wu, Igor Kiselev, Mrinmaya Sachan, Iryna Gurevych, Preslav Nakov, Timothy Baldwin, Artem Shelmanov
| Challenge: | Existing TTC scaling strategies and reasoning scorers are fragmented and evaluated under inconsistent protocols. |
| Approach: | They propose a framework for seamless test-time compute scaling of large language model reasoning . they use a modular Python library to implement state-of-the-art scaling strategy and scorer families . |
| Outcome: | The proposed framework evaluates performance and computational efficiency on mathematical and coding tasks. |
MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets (2021.findings-emnlp)
Copied to clipboard
Shraman Pramanick, Shivam Sharma, Dimitar Dimitrov, Md. Shad Akhtar, Preslav Nakov, Tanmoy Chakraborty
| Challenge: | a growing number of harmful memes are being used for trolling, cyberbullying and abuse . a new approach to detect harmful meme images and texts is emerging . |
| Approach: | They propose a multimodal deep neural network that detects harmful memes . they extend the recently released HarMeme dataset with additional memes and a new topic . |
| Outcome: | The proposed framework outperforms rival methods in detecting harmful memes and their target social entities. |
Instruction-Guided Poetry Generation in Arabic and Its Dialects (2026.findings-acl)
Copied to clipboard
Abdelrahman Sadallah, Kareem Elozeiri, Mervat Abassy, Rania Elbadry, Mohamed Anwar, Abed Alhakim Freihat, Preslav Nakov, Fajri Koto
| Challenge: | Existing literature on Arabic poetry has focused on analysis tasks such as interpretation or metadata prediction, e.g., rhyme schemes and titles. |
| Approach: | They propose to use a large-scale instruction-based dataset to generate Arabic poetry based on predefined criteria such as style and rhyme . |
| Outcome: | The proposed model can generate poetry that is aligned with user requirements, based on automated metrics and human evaluation with native Arabic speakers. |
A Survey of Confidence Estimation and Calibration in Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated impressive capabilities across a wide range of tasks in various domains, but they can be unreliable due to factual errors in their generations. |
| Approach: | They summarize recent advances in LLM confidence estimation and calibration and outline their main lessons learned. |
| Outcome: | The proposed methods can be used to assess the reliability of models and to calibrate them across tasks. |
One Size Does Not Fit All: Comparing NMT Representations of Different Granularities (N19-1)
Copied to clipboard
| Challenge: | Recent work has shown that contextualized word representations are a viable alternative to simple word prediction tasks. |
| Approach: | They propose to use subword units and characters to model morphology, syntax, and semantics instead of word embeddings. |
| Outcome: | The proposed representations are better for modeling syntax and more robust to noisy input. |
Characterizing the Entities in Harmful Memes: Who is the Hero, the Villain, the Victim? (2023.eacl-main)
Copied to clipboard
Shivam Sharma, Atharva Kulkarni, Tharun Suresh, Himanshi Mathur, Preslav Nakov, Md. Shad Akhtar, Tanmoy Chakraborty
| Challenge: | A common problem associated with meme comprehension lies in detecting the entities referenced and characterizing the role of each of these entities. |
| Approach: | They propose to use a memes dataset on US Politics and Covid-19 memes to characterize the role of harmful entities in memes. |
| Outcome: | The proposed model improves 4% over baseline and 1% over competing models. |
Exploring Language Model Generalization in Low-Resource Extractive QA (2025.coling-main)
Copied to clipboard
| Challenge: | Existing LLMs struggle with dataset demands of closed domains such as medicine and law . current LLM performance in closed domain is lacking, even on traditional tasks such as Natural Language Inference . |
| Approach: | They investigate Extractive Question Answering (EQA) with Large Language Models (LLMs) under domain drift . they find that LLMs struggle with dataset demands of closed domains . |
| Outcome: | The proposed model performs poorly in extractive question answering tasks under domain drift . the proposed model can generalize to domains that require specific knowledge without training . |
Multilingual Multifaceted Understanding of Online News in Terms of Genre, Framing, and Persuasion Techniques (2023.acl-long)
Copied to clipboard
| Challenge: | a new dataset of news articles is presented that covers genre, framing, and persuasion techniques. |
| Approach: | They propose a multilingual multifacet dataset of news articles annotated for genre, framing and persuasion techniques. |
| Outcome: | The proposed dataset contains 1,612 news articles covering recent news on current topics of public interest in six European languages. |
NewsClaims: A New Benchmark for Claim Detection from News with Attribute Knowledge (2022.emnlp-main)
Copied to clipboard
Revanth Gangi Reddy, Sai Chetan Chinthakindi, Zhenhailong Wang, Yi Fung, Kathryn Conger, Ahmed ELsayed, Martha Palmer, Preslav Nakov, Eduard Hovy, Kevin Small, Heng Ji
| Challenge: | Current claims detection methods focus on sentence analysis, ignoring other attributes . a key element of identifying misinformation is detecting the claims and the arguments that have been presented. |
| Approach: | They propose a benchmark for attribute-aware claim detection in the news domain . they extend the problem to include extraction of additional attributes related to each claim . |
| Outcome: | The proposed system performs well on the test, but human performance is still poor. |
Paper Bullets: Modeling Propaganda with the Help of Metaphor (2023.findings-eacl)
Copied to clipboard
| Challenge: | We hypothesize that it can be beneficial to model propaganda and metaphor together . we identify propaganda using loaded language and name-calling . |
| Approach: | They hypothesize that it can be beneficial to model propaganda and metaphor together . they use two datasets to identify propaganda techniques in news articles and memes . |
| Outcome: | The proposed model improves performance for the two most common propaganda techniques, especially loaded language and name-calling. |
Rethinking STS and NLI in Large Language Models (2024.findings-eacl)
Copied to clipboard
| Challenge: | Recent years have seen the rise of large language models (LLMs), where practitioners use task-specific prompts; this was shown to be effective for a variety of tasks. |
| Approach: | They propose to rethink semantic textual similarity (STS) and natural language inference (NLI) models with task-specific prompts and model overconfidence to capture disagreements between human judgements. |
| Outcome: | The proposed models are able to capture human opinions on individual examples without any parameter modifications. |
OpenFactCheck: Building, Benchmarking Customized Fact-Checking Systems and Evaluating the Factuality of Claims and LLMs (2025.coling-main)
Copied to clipboard
Yuxia Wang, Minghan Wang, Hasan Iqbal, Georgi N. Georgiev, Jiahui Geng, Iryna Gurevych, Preslav Nakov
| Challenge: | Large language models (LLMs) generate naturallysounding answers over a broad range of human inquiries, but they still produce content that deviates from real-world facts. |
| Approach: | They propose a framework for building customized automatic fact-checking systems, benchmarking their accuracy, evaluating factuality of LLMs, and verifying claims in a document. |
| Outcome: | The proposed framework assesses the factuality of free-form responses in open domains and evaluates factually of LLMs. |
Arabic Dataset for LLM Safeguard Evaluation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing studies on large language models have focused on English, but the safety of LLMs in Arabic remains under-explored. |
| Approach: | They propose to use Arabic-region-specific questions to evaluate LLMs' safety . they use a dual-perspective evaluation framework to examine differences between LLM responses . |
| Outcome: | The proposed framework assesses the LLM responses from both governmental and opposition viewpoints. |
FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoning (2026.acl-long)
Copied to clipboard
Zhuohan Xie, Daniil Orel, Rushil Thareja, Dhruv Sahnan, Hachem Madmoun, Fan Zhang, Debopriyo Banerjee, Georgi Nenkov Georgiev, Xueqing Peng, Lingfei Qian, Jimin Huang, Jinyan Su, Aaryamonvikram Singh, Rui Xing, Rania Elbadry, Chen Xu, Haonan Li, Fajri Koto, Ivan Koychev, Tanmoy Chakraborty, Yuxia Wang, Salem Lahlou, Veselin Stoyanov, Sophia Ananiadou, Preslav Nakov
| Challenge: | Existing benchmarks emphasize final numerical answers while neglecting intermediate reasoning steps. |
| Approach: | They propose a symbolic benchmark for verifiable Chain-of-Thought evaluation in finance . FINCHAIN spans 58 topics across 12 financial domains and three difficulty levels . |
| Outcome: | The proposed benchmark aims to bridge symbolic reasoning and factual verification. |
UNCERTAINTY-LINE: Length-Invariant Estimation of Uncertainty for Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) generate misleading or outright incorrect information. |
| Approach: | They propose a method that debiases uncertainty scores on output length and uses residuals as corrected, length-invariant estimates. |
| Outcome: | The proposed method improves over nominally length-normalized methods on machine translation, summarization, and question-answering tasks. |
M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection (2024.acl-long)
Copied to clipboard
Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Osama Mohammed Afzal, Tarek Mahmoud, Giovanni Puccetti, Thomas Arnold, Alham Aji, Nizar Habash, Iryna Gurevych, Preslav Nakov
| Challenge: | Large Language Models (LLMs) have brought an unprecedented surge in machine-generated text (MGT) societal implications are posed by their potential misuse and lack of training data. |
| Approach: | They propose a benchmark to detect machine-generated text in multiple languages . they use multi-domain and multi-generator corpus to identify which model generated the text . |
| Outcome: | The proposed benchmark compares a multilingual, multi-domain and multi-generator corpus of MGTs with human-generated content. |
DocScript: Document-level Script Event Prediction (2024.lrec-main)
Copied to clipboard
Puneet Mathur, Vlad I. Morariu, Aparna Garimella, Franck Dernoncourt, Jiuxiang Gu, Ramit Sawhney, Preslav Nakov, Dinesh Manocha, Rajiv Jain
| Challenge: | Existing script event prediction frameworks such as ChatGPT and FlanT5 lack the ability to learn long-range dependencies between events. |
| Approach: | They propose a novel script event prediction task which aims to predict the next event from a candidate list of narrative events in long-form documents. |
| Outcome: | The proposed architecture can learn sequential ordering between events at the document scale. |
QACheck: A Demonstration System for Question-Guided Multi-Hop Fact-Checking (2023.emnlp-demo)
Copied to clipboard
| Challenge: | Existing fact-checking systems lack transparency in their decision-making process, making it difficult for users to comprehend their reasoning process. |
| Approach: | They propose a Question-guided Multi-hop Fact-Checking system which asks a series of questions critical for verifying a claim. |
| Outcome: | The proposed model provides a comprehensive report detailing its reasoning process, guided by a sequence of questions, answer pairs, and the source of evidence supporting each question. |
A Survey on Stance Detection for Mis- and Disinformation Identification (2022.findings-naacl)
Copied to clipboard
| Challenge: | Understanding attitudes expressed in texts plays an important role in systems for detecting false information online, be it misinformation (unintentionally false) or disinformation (intentional false information). |
| Approach: | They examine the relationship between stance detection and mis- and disinformation detection online and examine the results of previous studies. |
| Outcome: | The proposed task is a component of fact-checking, rumour detection, and detecting previously fact- checked claims, and is compared with other related tasks such as argumentation mining and sentiment analysis. |
AICD Bench: A Challenging Benchmark for AI-Generated Code Detection (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for detecting AI-generated code are limited to binary human–machine classification under in-distribution settings. |
| Approach: | They propose to use AICD Bench to build a robust binary classification framework for large language models. |
| Outcome: | The proposed benchmark spans 2M examples, 77 models across 11 families, and 9 programming languages. |
SOLID: A Large-Scale Semi-Supervised Dataset for Offensive Language Identification (2021.findings-acl)
Copied to clipboard
| Challenge: | toxicity, hate speech, cyberbullying, and cyber-aggression are common themes in social media . authors present a dataset that is limited in size and biased towards offensive language . |
| Approach: | They present an expanded dataset that uses a taxonomy for offensive language identification . they show that using SOLID and OLID yields sizable performance gains . |
| Outcome: | The proposed dataset shows that it performs better than the OLID dataset for two different models. |
SAFARI: Cross-lingual Bias and Factuality Detection in News Media and News Articles (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a new corpus of news media and articles is developed to assess political bias and factuality in cross-lingual contexts . integrity and objectivity of news are crucial in an age of information sharing across cultural and language landscapes - a recent study shows . |
| Approach: | They propose a corpus of news media and articles for predicting political bias and factuality . they evaluate the cross-lingual ability of the models; however, they evaluate on English data . |
| Outcome: | The proposed corpus is unprecedented in its collection and evaluates on English data. |
Integrating Stance Detection and Fact Checking in a Unified Corpus (N18-2)
Copied to clipboard
| Challenge: | Existing methods for fact checking are not supported by existing datasets, which treat fact checking, document retrieval, source credibility, stance detection and rationale extraction as independent tasks. |
| Approach: | They propose to implement automatic fact checking on an Arabic fact checking corpus, which is the first of its kind. |
| Outcome: | The proposed approach is based on an Arabic fact checking corpus, the first of its kind. |
Entity Framing and Role Portrayal in the News (2025.findings-acl)
Copied to clipboard
Tarek Mahmoud, Zhuohan Xie, Dimitar Iliyanov Dimitrov, Nikolaos Nikolaidis, Purificação Silvano, Roman Yangarber, Shivam Sharma, Elisa Sartori, Nicolas Stefanovitch, Giovanni Da San Martino, Jakub Piskorski, Preslav Nakov
| Challenge: | a dataset of news articles containing 22 fine-grained characters is annotated for entity framing and role portrayal . the dataset includes 1,378 recent news articles in five languages focusing on the Ukraine-Russia War and climate change . |
| Approach: | They propose a multilingual and hierarchical corpus annotated for entity framing and role portrayal in news articles. |
| Outcome: | The proposed dataset includes 1,378 recent news articles in five languages focusing on the Ukraine-Russia War and climate change . the authors report evaluation results on state-of-the-art multilingual transformers and hierarchical zero-shot learning using LLMs at the level of a document, paragraph, and sentence . |
DISARM: Detecting the Victims Targeted by Harmful Memes (2022.findings-naacl)
Copied to clipboard
| Challenge: | DISARM is a framework that uses named-entity recognition and person identification to detect all entities a meme is referring to and then incorporates a novel contextualized deep neural network to classify whether the meme intends to harm these entities. |
| Approach: | They propose a framework that uses named-entity recognition and person identification to detect all entities a meme is referring to and incorporates a novel contextualized deep neural network to classify whether the meme intends to harm them. |
| Outcome: | The proposed framework outperforms 10 unimodal and multimodal systems and reduces error rate of harmful target identification by 9 % absolute over baseline systems. |
Loki: An Open-Source Tool for Fact Verification (2025.coling-demos)
Copied to clipboard
Haonan Li, Xudong Han, Hao Wang, Yuxia Wang, Minghan Wang, Rui Xing, Yilin Geng, Zenan Zhai, Preslav Nakov, Timothy Baldwin
| Challenge: | Loki is an open-source fact-checking tool designed to address the growing problem of misinformation. |
| Approach: | They propose a tool that breaks down the fact-checking task into five steps . they propose LOKI, which offers a semiautomated, human-in-the-loop approach . |
| Outcome: | a new open-source tool is designed to address the growing problem of misinformation . the tool breaks down the fact-checking task into five steps to assist human judgment . |
Predicting Factuality of Reporting and Bias of News Media Sources (D18-1)
Copied to clipboard
| Challenge: | a new study examines the factuality of news media and its biases . social media has democratized content creation and spread information online . |
| Approach: | They propose to characterize entire news media to predict factuality and bias . they experiment with news websites and a set of features derived from their content . |
| Outcome: | The proposed model shows that the features of news websites perform better than baseline . the results show that the feature types are important for fact-checking systems . |
bgGLUE: A Bulgarian General Language Understanding Evaluation Benchmark (2023.acl-long)
Copied to clipboard
Momchil Hardalov, Pepa Atanasova, Todor Mihaylov, Galia Angelova, Kiril Simov, Petya Osenova, Veselin Stoyanov, Ivan Koychev, Preslav Nakov, Dragomir Radev
| Challenge: | bgGLUE is a benchmark for evaluating language models on natural language understanding (NLU) tasks in Bulgarian. |
| Approach: | They propose to use a benchmark to evaluate language models on NLU tasks in Bulgarian. |
| Outcome: | The proposed model performs well on sequence labeling tasks, but there is room for improvement for tasks that require more complex reasoning. |
KazMMLU: Evaluating Language Models on Kazakh, Russian, and Regional Knowledge of Kazakhstan (2025.acl-long)
Copied to clipboard
Mukhammed Togmanov, Nurdaulet Mukhituly, Diana Turmakhan, Jonibek Mansurov, Maiya Goloburda, Akhmed Sakip, Zhuohan Xie, Yuxia Wang, Bekassyl Syzdykov, Nurkhan Laiyk, Alham Fikri Aji, Ekaterina Kochmar, Preslav Nakov, Fajri Koto
| Challenge: | Kazakh language remains underrepresented in the field of natural language processing despite the country's population exceeding twenty million . however, there is a lack of dedicated models and benchmark evaluations specifically tailored to Kazakh languages. |
| Approach: | They propose to create a dataset specifically designed for Kazakh language with 23,000 questions sourced from authentic educational materials and manually validated by native speakers and educators. |
| Outcome: | The first MMLU-style dataset specifically designed for Kazakh language. |
From Generation to Detection: A Multimodal Multi-Task Dataset for Benchmarking Health Misinformation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Infodemics and health misinformation have significant negative impact on individuals and society . generative AI has significantly accelerated the spread and expanded the reach of health misinfo . |
| Approach: | MM-Health is a large scale multimodal misinformation dataset in the health domain . it includes human-generated multimodal information and AI-generated multiplemodal information . |
| Outcome: | MM-Health is a large scale misinformation dataset in the health domain . it includes human-generated multimodal information and AI-generated content . |
Aristotle: Mastering Logical Reasoning with A Logic-Complete Decompose-Search-Resolve Framework (2025.acl-long)
Copied to clipboard
Jundong Xu, Hao Fei, Meng Luo, Qian Liu, Liangming Pan, William Yang Wang, Preslav Nakov, Mong-Li Lee, Wynne Hsu
| Challenge: | Existing systems fail to fully leverage the structure of logical tasks throughout the reasoning process, causing bottlenecks in efficiency and efficacy. |
| Approach: | They propose a logic-complete reasoning framework, Aristotle, which integrates symbolic expressions and logical rules into the entire reasoning process. |
| Outcome: | The proposed framework outperforms state-of-the-art reasoning frameworks in accuracy and efficiency. |
Unconditional Truthfulness: Learning Unconditional Uncertainty of Large Language Models (2025.emnlp-main)
Copied to clipboard
Artem Vazhentsev, Ekaterina Fadeeva, Rui Xing, Gleb Kuzmin, Ivan Lazichny, Alexander Panchenko, Preslav Nakov, Timothy Baldwin, Maxim Panov, Artem Shelmanov
| Challenge: | Uncertainty quantification (UQ) is a promising approach for detecting hallucinations and low-quality outputs of Large Language Models (LLMs). |
| Approach: | They propose to learn conditional dependency between autoregressive LLM generation steps from attention-based features and a two-staged training procedure to incorporate recurrent features. |
| Outcome: | The proposed method is highly effective for selective generation, achieving substantial improvements over rivaling unsupervised and supervised approaches. |
A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputs (2025.emnlp-main)
Copied to clipboard
Artem Shelmanov, Ekaterina Fadeeva, Akim Tsvigun, Ivan Tsvigun, Zhuohan Xie, Igor Kiselev, Nico Daheim, Caiqi Zhang, Artem Vazhentsev, Mrinmaya Sachan, Preslav Nakov, Timothy Baldwin
| Challenge: | Uncertainty quantification (UQ) is a framework for assessing the reliability of model outputs. |
| Approach: | They introduce pre-trained UQ heads for LLMs that are highly robust and generalized to languages they were not explicitly trained on. |
| Outcome: | The pre-trained heads significantly improve their ability to capture uncertainty compared to unsupervised methods. |
FinCARDS: Card-Based Analyst Reranking for Financial Document Question Answering (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing reranking frameworks optimize semantic relevance, leading to unstable rankings and opaque decisions on long documents. |
| Approach: | They propose a structured reranking framework that reframes financial evidence selection as constraint satisfaction under a finance-aware schema. |
| Outcome: | FINCARDS improves early-rank retrieval over lexical and LLM-based reranking baselines while reducing ranking variance. |
On the Risk of Misinformation Pollution with Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study demonstrates that large language models can be misused for generating credible-sounding misinformation . however, the ability to produce credible text raises concerns regarding their potential misuse . |
| Approach: | They propose three defense strategies to mitigate misinformation generated by Large Language Models . they propose a threat model and simulate potential misuse scenarios . |
| Outcome: | The proposed defense strategies have shown promising results, albeit with costs. |
Saliency-Aware Interpolative Augmentation for Multimodal Financial Prediction (2024.lrec-main)
Copied to clipboard
Samyak Jain, Parth Chhabra, Atula Tejaswi Neerkaje, Puneet Mathur, Ramit Sawhney, Shivam Agarwal, Preslav Nakov, Sudheer Chava, Dinesh Manocha
| Challenge: | Recent advances in the Financial AI realm have expanded the scope of data and methods they use, such as textual and audio cues from financial earnings calls, but limitations exist. |
| Approach: | They propose a Saliency-guided Hierarchical Mixup augmentation technique for multimodal financial prediction tasks. |
| Outcome: | The proposed technique outperforms state-of-the-art methods by 3-7% on financial earnings and conference call datasets. |
EXAMS: A Multi-subject High School Examinations Dataset for Cross-lingual and Multilingual Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | EXAMS is a benchmark dataset for cross-lingual and multilingual question answering for high school examinations. |
| Approach: | They propose to use EXAMS to evaluate cross-lingual and multilingual question answering for high school examinations. |
| Outcome: | The proposed model can be used to explore multilingual reasoning and knowledge transfer methods and pre-trained models in schools in different languages, which was not possible by now. |
Is Human-Like Text Liked by Humans? Multilingual Human Detection and Preference Against AI (2026.acl-long)
Copied to clipboard
Yuxia Wang, Rui Xing, Jonibek Mansurov, Giovanni Puccetti, Zhuohan Xie, Minh Ngoc Ta, Jiahui Geng, Jinyan Su, Mervat Abassy, Saadeldine Eletter, Kareem Elozeiri, Nurkhan Laiyk, Maiya Goloburda, Tarek Mahmoud, Raj Vardhan Tomar, Alexander Aziz, Ryuto Koike, Masahiro Kaneko, Artem Shelmanov, Ekaterina Artemova, Vladislav Mikhailov, Akim Tsvigun, Alham Fikri Aji, Nizar Habash, Iryna Gurevych, Preslav Nakov
| Challenge: | Prior studies have shown that distinguishing text generated by Large Language Models from human-written text is challenging for humans and often no better than random guessing. |
| Approach: | They conduct extensive case study to determine the upper bound of human detection accuracy. |
| Outcome: | The findings challenge previous conclusions on human detection accuracy across languages and domains. |
Can Machines Resonate with Humans? Evaluating the Emotional and Empathic Comprehension of LMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Empathy plays a pivotal role in fostering prosocial behavior, often triggered by the sharing of personal experiences through narratives. |
| Approach: | They propose to use contrastive learning with masked LMs and supervised fine-tuning with large language models to improve empathy understanding in NLP models. |
| Outcome: | The proposed methods show that there is low agreement among annotators and that cultural differences are a factor in their interpretation of empathy. |
LLM-DetectAIve: a Tool for Fine-Grained Machine-Generated Text Detection (2024.emnlp-demo)
Copied to clipboard
Mervat Abassy, Kareem Elozeiri, Alexander Aziz, Minh Ta, Raj Tomar, Bimarsha Adhikari, Saad Ahmed, Yuxia Wang, Osama Mohammed Afzal, Zhuohan Xie, Jonibek Mansurov, Ekaterina Artemova, Vladislav Mikhailov, Rui Xing, Jiahui Geng, Hasan Iqbal, Zain Mujahid, Tarek Mahmoud, Akim Tsvigun, Alham Aji, Artem Shelmanov, Nizar Habash, Iryna Gurevych, Preslav Nakov
| Challenge: | a large number of machine-generated texts are often hard to distinguish between human-written and machine-generated text . this raises concerns about potential misuse, especially within educational and academic domains . |
| Approach: | They propose a system that can detect whether a text is human-written or machine-generated . they use a fine-grained classification schema to identify the use of machine-generated text . |
| Outcome: | The proposed system can distinguish between human-written and machine-generated text . it can detect attempts to obfuscate the fact that a text was machine- generated . |
Detecting Propaganda Techniques in Memes (2021.acl-long)
Copied to clipboard
Dimitar Dimitrov, Bishr Bin Ali, Shaden Shaar, Firoj Alam, Fabrizio Silvestri, Hamed Firooz, Preslav Nakov, Giovanni Da San Martino
| Challenge: | Propaganda can be defined as a form of communication that aims to influence opinions or the actions of people towards a specific goal. |
| Approach: | They propose to detect the type of propaganda techniques used in memes by annotating them with 22 techniques. |
| Outcome: | The proposed model identifies 22 propaganda techniques in memes, which can appear in text, image or both . |
Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkers (2024.findings-emnlp)
Copied to clipboard
Yuxia Wang, Revanth Gangi Reddy, Zain Mujahid, Arnav Arora, Aleksandr Rubashevskii, Jiahui Geng, Osama Mohammed Afzal, Liangming Pan, Nadav Borenstein, Aditya Pillai, Isabelle Augenstein, Iryna Gurevych, Preslav Nakov
| Challenge: | Large language models generate naturally sounding answers over a broad range of human inquiries, but they often generate answers that contradict real-world facts. |
| Approach: | They propose a framework for annotating and evaluating the factuality of large language models . they propose 'factcheck-bench' which provides a multi-stage annotation scheme . |
| Outcome: | The proposed framework outperforms several popular LLM fact-checkers in claim, sentence, and document levels. |
Fine-Grained Analysis of Propaganda in News Article (D19-1)
Copied to clipboard
| Challenge: | Existing methods for detecting propaganda are noisy and lack of explainability. |
| Approach: | They propose to perform fine-grained analysis of texts by detecting all fragments that contain propaganda techniques as well as their type. |
| Outcome: | The proposed model outperforms several strong BERT-based baselines. |
PolyNarrative: A Multilingual, Multilabel, Multi-domain Dataset for Narrative Extraction from News Articles (2025.acl-long)
Copied to clipboard
Nikolaos Nikolaidis, Nicolas Stefanovitch, Purificação Silvano, Dimitar Iliyanov Dimitrov, Roman Yangarber, Nuno Guimarães, Elisa Sartori, Ion Androutsopoulos, Preslav Nakov, Giovanni Da San Martino, Jakub Piskorski
| Challenge: | a new dataset of news articles annotated for narratives provides a framework for narrative detection . recurring narratives can propagate with very high velocity across audiences, languages and countries . |
| Approach: | They propose a multilingual dataset annotated for narratives using two-level taxonomies . they define narrative as a recurring, repetitive, overt or implicit claim that promotes a specific interpretation or viewpoint on an ongoing topic . |
| Outcome: | The proposed dataset will foster research in narrative detection and enable new research directions . the authors identify multiple narratives in the same article, and the results are published online . |
Annotating the Annotators: Analysis, Insights and Modelling from an Annotation Campaign on Persuasion Techniques Detection (2025.findings-acl)
Copied to clipboard
Davide Bassi, Dimitar Iliyanov Dimitrov, Bernardo D’Auria, Firoj Alam, Maram Hasanain, Christian Moro, Luisa Orrù, Gian Piero Turchi, Preslav Nakov, Giovanni Da San Martino
| Challenge: | Existing annotation campaigns based on heuristic guidelines have not been thoroughly discussed. |
| Approach: | They propose a probabilistic model for optimizing intervention scheduling to reduce the cost of an expert oversight in annotation tasks. |
| Outcome: | The proposed model advocates for an expert oversight in annotation tasks and periodic quality audits to reduce costs. |
DocEdit-v2: Document Structure Editing Via Multimodal LLM Grounding (2024.emnlp-main)
Copied to clipboard
Manan Suri, Puneet Mathur, Franck Dernoncourt, Rajiv Jain, Vlad Morariu, Ramit Sawhney, Preslav Nakov, Dinesh Manocha
| Challenge: | Document structure editing involves manipulating localized textual, visual, and layout components in document images based on user’s requests. |
| Approach: | They propose a framework that performs end-to-end document editing by leveraging Large Multimodal Models (LMMs) by localizing edit regions of interest and disambiguating user edit requests into edit commands. |
| Outcome: | The proposed framework outperforms baselines on edit command generation (2-33%), RoI bounding box detection (12-31%), and overall document editing (1-12%) tasks. |
MGM: Global Understanding of Audience Overlap Graphs for Predicting the Factuality and the Bias of News Media (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for profiling news media focus on textual features, causing them to overlook complex relationships between entities. |
| Approach: | They propose a framework for profiling news media from the lens of political bias and factuality. |
| Outcome: | The proposed framework improves existing models and improves them by integrating structural information from similar nodes. |
Stereotype Bias in a Bilingual Setting: A Culturally Grounded Evaluation in Kazakhstan (2026.acl-long)
Copied to clipboard
Nurkhan Laiyk, Daniil Orel, Ayana Mussabayeva, Maiya Goloburda, Kamila Kuishibekova, Liya Goloburda, Diana Turmakhan, Preslav Nakov, Yuxia Wang, Fajri Koto
| Challenge: | Stereotype bias in language models is largely understudied in English . language models perform strongly on downstream NLP tasks, but they are pre-trained on large text corpora . |
| Approach: | They use a dataset to assess stereotype bias in language models in Kazakhstan . they find that stereotype bias is most pronounced in code-mixed inputs . |
| Outcome: | The proposed dataset shows that stereotype bias is most pronounced in code-mixed inputs. |
NarratEX Dataset: Explaining the Dominant Narratives in News Texts (2025.findings-emnlp)
Copied to clipboard
Nuno Guimarães, Purificação Silvano, Ricardo Campos, Alipio Jorge, Ana Filipa Pacheco, Dimitar Iliyanov Dimitrov, Nikolaos Nikolaidis, Roman Yangarber, Elisa Sartori, Nicolas Stefanovitch, Preslav Nakov, Jakub Piskorski, Giovanni Da San Martino
| Challenge: | a dataset is created to explain the choice of the dominant narrative in a news article . the dataset is intended to address discourse polarization and propaganda detection . |
| Approach: | They propose a dataset for explaining the choice of the dominant narrative in a news article . the dataset is annotated manually with a dominant narrative and sub-narrative labels . |
| Outcome: | The proposed dataset is designed to explain the choice of the dominant narrative in a news article. |
Do-Not-Answer: Evaluating Safeguards in LLMs (2024.findings-eacl)
Copied to clipboard
| Challenge: | a dataset evaluating harmful capabilities in large language models is available at https://github.com/Libr-AI/do-not-answer. |
| Approach: | They collect an open-source dataset to evaluate the safeguards in large language models . they find that simple BERT-style classifiers can achieve results comparable to GPT-4 . |
| Outcome: | The proposed dataset compares the safety of six popular LLMs to GPT-4 on automatic safety evaluation. |
Lost in Translation, Found in Spans: Identifying Claims in Multilingual Social Media (2023.emnlp-main)
Copied to clipboard
| Challenge: | Claim span identification (CSI) is an important step in fact-checking pipelines . despite its importance to journalists and fact-seekers, it remains a understudied problem . |
| Approach: | They propose to use social media claims to identify text segments that contain a check-worthy claim or assertion in a social media post. |
| Outcome: | The proposed dataset outperforms other cross-lingual transfer methods on multiple languages. |
SAHM: A Benchmark for Arabic Financial and Shari’ah-Compliant Reasoning (2026.acl-long)
Copied to clipboard
Rania Elbadry, Sarfraz Ahmad, Ahmed Heakl, Dani Bouch, Momina Ahsan, Muhra AlMahri, Marwa Elsaid Khalil, Yuxia Wang, Salem Lahlou, Sophia Ananiadou, Veselin Stoyanov, Jimin Huang, Xueqing Peng, Preslav Nakov, Zhuohan Xie
| Challenge: | English financial NLP has progressed rapidly through benchmarks for sentiment, document understanding, and financial question answering. |
| Approach: | They propose a document-grounded benchmark and instruction-tuning dataset for Arabic financial NLP and Shari’ah-compliant reasoning. |
| Outcome: | The proposed dataset contains 14,380 expert-verified instances spanning seven tasks . it includes financial sentiment analysis, extractive summarization, and event–cause reasoning . |
Prta: A System to Support the Analysis of Propaganda Techniques in the News (2020.acl-demos)
Copied to clipboard
Giovanni Da San Martino, Shaden Shaar, Yifan Zhang, Seunghak Yu, Alberto Barrón-Cedeño, Preslav Nakov
| Challenge: | recent events have brought the public attention to the dangers of online disinformation. |
| Approach: | a new tool helps users analyze propaganda using specific rhetorical and psychological techniques. a prta system identifies the spans in which propaganda techniques occur and compares them. |
| Outcome: | a new tool can analyze articles crawled on a regular basis and compare them on the basis of their use of propaganda techniques. |
Toxicity Red-Teaming: Benchmarking LLM Safety in Singapore’s Low-Resource Languages (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have transformed natural language processing, but their safety mechanisms remain under-explored in low-resource, multilingual settings. |
| Approach: | They propose a red-teaming approach to probe LLM vulnerabilities in Singapore's diverse linguistic context using a dataset and evaluation framework. |
| Outcome: | The proposed framework systematically probes LLM vulnerabilities in three real-world scenarios including Singlish, Chinese, Malay, and Tamil. |
Data-Informed Global Sparseness in Attention Mechanisms for Deep Neural Networks (2024.lrec-main)
Copied to clipboard
| Challenge: | Attention pruning techniques have been developed to identify and exploit sparseness . previous work has taken pioneering steps to discover and explain the sparsity in attention patterns . |
| Approach: | They propose a framework that observes attention patterns in a fixed dataset and generates a global sparseness mask. |
| Outcome: | The proposed approach saves 90% of computations and maintains quality of results. |
FinReporting: An Agentic Workflow for Localized Reporting of Cross-Jurisdiction Financial Disclosure (2026.acl-demo)
Copied to clipboard
Fan Zhang, Mingzi Song, Rania Elbadry, Yankai Chen, Shaobo Wang, Yixi Zhou, Xunwen Zheng, Yueru He, Yuyang Dai, Georgi Nenkov Georgiev, Ayesha Gull, Muhammad Usman Safder, Fan Wu, Liyuan Meng, Fengxian Ji, Junning Zhao, Xueqing Peng, Jimin Huang, YU Chen, Xue Liu, Preslav Nakov, Zhuohan Xie
| Challenge: | FinReporting is an agentic workflow for localized cross-jurisdiction financial reporting . existing approaches assume a single-market setting and overlook structural differences across jurisdictions . |
| Approach: | They propose a workflow that decomposes financial reporting into auditable stages . they use Large Language Models to extract and summarize corporate disclosures . |
| Outcome: | The proposed system decomposes reporting into auditable stages . it improves consistency and reliability under heterogeneous reporting regimes. |
Contrastive Language Adaptation for Cross-Lingual Stance Detection (D19-1)
Copied to clipboard
| Challenge: | Current approaches to fact-checking are time-consuming and tedious. |
| Approach: | They propose a novel approach which leverages labeled data in one language to identify relative perspective of a document with respect to a claim in a different target language. |
| Outcome: | The proposed approach can deal with the challenge of limited labeled data in the target language. |
ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic (2024.findings-acl)
Copied to clipboard
Fajri Koto, Haonan Li, Sara Shatnawi, Jad Doughman, Abdelrahman Sadallah, Aisha Alraeesi, Khalid Almubarak, Zaid Alyafeai, Neha Sengupta, Shady Shehata, Nizar Habash, Preslav Nakov, Timothy Baldwin
| Challenge: | evaluating language models in Arabic remains challenging due to limited datasets . focus has shift to reasoning and knowledge-intensive tasks due to lack of relevant datasets. |
| Approach: | They propose to use ArabicMMLU to evaluate models' understanding of Arabic . they use 40 tasks and 14,575 multiple-choice questions from school exams in different countries . |
| Outcome: | The ArabicMMLU is the first multi-task language understanding benchmark for the Arabic language . it is based on 40 tasks and 14,575 multiple-choice questions in modern standard Arabic . the models are based in different countries across North Africa, the Levant, and the Gulf regions . |
UrduFactCheck: An Agentic Fact-Checking Framework for Urdu with Evidence Boosting and Benchmarking (2025.findings-emnlp)
Copied to clipboard
Sarfraz Ahmad, Hasan Iqbal, Momina Ahsan, Numaan Naeem, Muhammad Ahsan Riaz Khan, Arham Riaz, Muhammad Arslan Manzoor, Yuxia Wang, Preslav Nakov
| Challenge: | Existing automated fact-checking systems are predominantly developed for English . Existing systems focus on claim verification, but UrduFactQA targets factuality . |
| Approach: | They propose two hand-annotated benchmarks to enable fact-checking and factual consistency evaluation in Urdu. |
| Outcome: | The proposed benchmarks are the first of their kind for Urdu and are available online. |
CrowdChecked: Detecting Previously Fact-Checked Claims in Social Media (2022.aacl-main)
Copied to clipboard
| Challenge: | Existing systems to automate fact-checking lack credibility in the eyes of the users. |
| Approach: | They propose to perform automatic fact-checking by verifying whether an input claim has been fact- checked by professional fact- checkers and to return back an article that explains their decision. |
| Outcome: | The proposed method improves on the CLEF’21 CheckThat! test set by two points absolute. |
RISE: Robust Early-exiting Internal Classifiers for Suicide Risk Evaluation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing systems for risk assessment are prone to incorrectly predicting risk severity and have no early detection mechanisms. |
| Approach: | They propose a novel mechanism for accurate early detection of suicide risk by ensembling Hyperbolic Internal Classifiers equipped with an abstention mechanism and early exit inference capabilities. |
| Outcome: | The proposed model abstains from 84% incorrect predictions on Reddit data while out-predicting state of the art models upto 3.5x earlier. |
A Neighborhood Framework for Resource-Lean Content Flagging (2022.tacl-1)
Copied to clipboard
Sheikh Muhammad Sarwar, Dimitrina Zlatkova, Momchil Hardalov, Yoan Dinkov, Isabelle Augenstein, Preslav Nakov
| Challenge: | Existing approaches to cross-lingual content flagging with limited target language data are lacking in many languages. |
| Approach: | They propose a framework for cross-lingual content flagging with limited target- language data based on a nearest-neighbor architecture and a transformer representation in all its components. |
| Outcome: | The proposed framework outperforms previous work in terms of predictive performance on eight languages from two different datasets. |
Tanbih: Get To Know What You Are Reading (D19-3)
Copied to clipboard
Yifan Zhang, Giovanni Da San Martino, Alberto Barrón-Cedeño, Salvatore Romeo, Jisun An, Haewoon Kwak, Todor Staykovski, Israa Jaradat, Georgi Karadzhov, Ramy Baly, Kareem Darwish, James Glass, Preslav Nakov
| Challenge: | Nowadays, more and more readers consume news online. |
| Approach: | They propose a news platform that displays news grouped into events and generates media profiles that show the general factuality of reporting, the degree of propagandistic content, hyper-partisanship, leading political ideology, general frame of reporting and stance with respect to various claims and topics of a media outlet. |
| Outcome: | The proposed news platform displays news grouped into events and generates media profiles that show the factuality of reporting, the degree of propagandistic content, hyper-partisanship, leading political ideology, general frame of reporting and stance with respect to various claims and topics of a news outlet. |
FIRE: Fact-checking with Iterative Retrieval and Verification (2025.findings-naacl)
Copied to clipboard
Zhuohan Xie, Rui Xing, Yuxia Wang, Jiahui Geng, Hasan Iqbal, Dhruv Sahnan, Iryna Gurevych, Preslav Nakov
| Challenge: | Fact-checking long-form text is challenging, and breaking it down into multiple atomic claims is not cost-effective. |
| Approach: | They propose a novel agent-based framework that integrates evidence retrieval and claim verification in an iterative manner. |
| Outcome: | The proposed framework reduces large language model (LLM) costs by an average of 7.6 times and search costs by 16.5 times while retaining the same performance. |
A Multi-View Media Profiling Suite: Resources, Evaluation, and Analysis (2026.findings-acl)
Copied to clipboard
Muhammad Arslan Manzoor, Dilshod Azizov, Daniil Orel, Umer Siddique, Zain Muhammad Mujahid, Yufang Hou, Preslav Nakov
| Challenge: | a large-scale label set for media outlets from Media Bias/Fact Check (MBFC) is lacking in the field. |
| Approach: | They propose to use a large-scale label set to analyze outlets' representations . they also propose to evaluate embedding views and fusion strategies . |
| Outcome: | The proposed method achieves state-of-the-art results on ACL-2020 and establishes strong benchmarks on MBFC-2025. |
CoDet-M4: Detecting Machine-Generated Code in Multi-Lingual, Multi-Generator and Multi-Domain Settings (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have revolutionized code generation but have significant consequences for programming skills, ethics, and assessment integrity. |
| Approach: | They propose a framework capable of distinguishing between human-written and LLM-generated program code across multiple programming languages, code generators, and domains. |
| Outcome: | The proposed framework distinguishes between human-written and LLM-generated program code across multiple programming languages, code generators, and domains. |
A Template Is All You Meme (2025.naacl-long)
Copied to clipboard
| Challenge: | Templatic memes are a form of communication capable of succinctly conveying complicated messages. |
| Approach: | They propose a method to match memes to a knowledge base of 5,200 meme templates and 54,000 examples of template instances using a distance-based lookup. |
| Outcome: | The proposed method improves general meme knowledge and sample efficiency, leading to more robust models. |
PropaInsight: Toward Deeper Understanding of Propaganda in Terms of Techniques, Appeals, and Intent (2025.coling-main)
Copied to clipboard
Jiateng Liu, Lin Ai, Zizhou Liu, Payam Karisani, Zheng Hui, Yi Fung, Preslav Nakov, Julia Hirschberg, Heng Ji
| Challenge: | Existing research on propaganda detection does not capture the motives behind the content or its broader impact. |
| Approach: | They propose a framework that dissects propaganda into techniques, arousal appeals, and underlying intent. |
| Outcome: | The proposed framework improves performance in a wide range of scenarios and can be used to identify and categorize propaganda techniques. |
The Role of Context in Detecting Previously Fact-Checked Claims (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent years have seen the proliferation of disinformation and fake news online. |
| Approach: | They propose to model the context of a political debate and the contexts of the document describing the fact-checked claim. |
| Outcome: | The proposed model improves on the state-of-the-art model by modeling the context of the claim . the experimental results show that the model can provide 10+ points of improvement over the state of the art model . |
Assisting the Human Fact-Checkers: Detecting All Previously Fact-Checked Claims in a Document (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent years have brought us a proliferation of false claims online, which spread fast . fact-checkers have been using automated fact-finding to verify claims . |
| Approach: | They propose a system that can detect claims that can be fact-checked by a given database . they create a manually annotated document dataset and propose evaluation measures . |
| Outcome: | The proposed system achieves sizable performance gains over strong baselines. |
Instruction Tuning on Public Government and Cultural Data for Low-Resource Language: a Case Study in Kazakh (2025.acl-long)
Copied to clipboard
| Challenge: | Instruction tuning in low-resource languages remains underexplored due to limited text data, particularly in government and cultural domains. |
| Approach: | They propose to open-source a large-scale instruction-following dataset covering key institutional and cultural knowledge relevant to Kazakhstan. |
| Outcome: | The proposed dataset improves LLMs’ understanding of procedural, legal, and structural governance topics. |
Cross-Domain Label-Adaptive Stance Detection (2021.emnlp-main)
Copied to clipboard
| Challenge: | Stance detection is a task that focuses on the classification of a writer’s viewpoint towards a target. |
| Approach: | They propose an end-to-end unsupervised framework for out-of-domain prediction of unseen, user-defined labels. |
| Outcome: | The proposed framework shows that it can be used to predict unseen labels over strong baselines. |
PASTA: Table-Operations Aware Fact Verification via Sentence-Table Cloze Pre-training (2022.emnlp-main)
Copied to clipboard
| Challenge: | Table-based fact verification has attracted a lot of attention recently due to the lack of datasets that can be used to pre-train language models to be aware of common table operations. |
| Approach: | They propose a table-based fact verification tool that pre-trains language models to be aware of common table operations such as aggregating a column or comparing tuples. |
| Outcome: | The proposed method outperforms previous methods on two table-based fact verification datasets TabFact and SEM-TAB- FACTS. |
Joint Multitask Learning for Community Question Answering Using Task-Specific Embeddings (D18-1)
Copied to clipboard
| Challenge: | Stack-Overflow, Quora, and Yahoo! Answers forums are not moderated, which results in noisy and redundant content. |
| Approach: | They use deep neural networks to learn meaningful task-specific embeddings . they incorporate the embeddables into a conditional random field model . |
| Outcome: | The proposed task improves significantly across evaluation metrics. |
Unleashing the Power of Discourse-Enhanced Transformers for Propaganda Detection (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing systems focused on the surface words, ignoring the linguistic structure of the texts. |
| Approach: | They propose to use discourse analysis to analyze paragraph-level and token-level classifications and propose a Transformer architecture that can be used to detect propaganda. |
| Outcome: | The proposed system improves on English and Russian texts and shows strong correlations between propaganda instances and discourse spans. |
ExaGPT: Example-Based Machine-Generated Text Detection for Human Interpretability (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing interpretable detectors are not aligned with the human decision-making process and fail to offer evidence that users easily understand. |
| Approach: | They propose an interpretable detection approach that checks whether a text is human-written or LLM-generated by checking whether it shares more similar spans with human-generated texts. |
| Outcome: | ExaGPT outperforms interpretable detectors by +37.0 points at a false positive rate of 1%. |
Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification (2024.findings-acl)
Copied to clipboard
Ekaterina Fadeeva, Aleksandr Rubashevskii, Artem Shelmanov, Sergey Petrakov, Haonan Li, Hamdy Mubarak, Evgenii Tsymbalov, Gleb Kuzmin, Alexander Panchenko, Timothy Baldwin, Preslav Nakov, Maxim Panov
| Challenge: | Large language models are notorious for producing erroneous claims in their output. |
| Approach: | They propose a fact-checking and hallucination detection pipeline based on token-level uncertainty quantification that removes the impact of uncertainty about what claim to generate on the current step and what surface form to use. |
| Outcome: | The proposed method can fact-check the atomic claims in the output of large language models. |
TART: An Open-Source Tool-Augmented Framework for Explainable Table-based Reasoning (2025.findings-naacl)
Copied to clipboard
| Challenge: | Current Large Language Models lack ability to understand table structures and apply precise numerical reasoning. |
| Approach: | They propose a tool-augmented reasoning framework for table-based tasks that integrates LLMs with specialized tools. |
| Outcome: | The proposed framework improves on the TOOLTAB dataset, a benchmark for LLMs in table–tool integration. |
RuleBERT: Teaching Soft Rules to Pre-Trained Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) are limited in their ability to capture and use common-sense knowledge. |
| Approach: | They propose to teach PLMs how to reason with soft Horn rules by leveraging logical rules to learn how to predict precise probabilities. |
| Outcome: | The proposed model performs well on logical rules that were unseen at training. |
Evaluating Pronominal Anaphora in Machine Translation: An Evaluation Measure and a Test Suite (D19-1)
Copied to clipboard
| Challenge: | Currently, machine translation is performed at the level of individual sentences, in isolation from the rest of the document. |
| Approach: | They propose a dataset that can be used as a test suite for pronoun translation . they propose an evaluation measure to differentiate good and bad pronounce translations . |
| Outcome: | The proposed dataset can be used as a test suite for pronoun translation in English . it covers multiple source languages and different pronouner errors drawn from real system translations . |
OpenFactCheck: A Unified Framework for Factuality Evaluation of LLMs (2024.emnlp-demo)
Copied to clipboard
| Challenge: | Large language models (LLMs) often produce content that deviates from real-world facts. |
| Approach: | They developed a unified framework to assess the factuality of large language models . open-sourced framework is publicly available as a Python library and web service . |
| Outcome: | OpenFactCheck is open-sourced and publicly released as a Python library and web service. |
Profiling News Media for Factuality and Bias Using LLMs and the Fact-Checking Methodology of Human Experts (2025.findings-acl)
Copied to clipboard
| Challenge: | Important efforts to characterize news media outlets in terms of their political bias and factuality are labor-intensive and prone to human biases. |
| Approach: | They propose a method that emulates criteria used by professional fact-checkers to assess the factuality and political bias of an entire outlet. |
| Outcome: | The proposed method improves on baselines and with multiple LLMs. |
When Personalization Tricks Detectors: The Feature-Inversion Trap in Machine-Generated Text Detection (2026.acl-long)
Copied to clipboard
Lang Gao, Xuhui Li, Chenxi Wang, Mingzhe Li, Wei Liu, Zirui Song, Jinghui Zhang, Rui Yan, Preslav Nakov, Xiuying Chen
| Challenge: | Personalized MGT detection remains largely underexplored due to personalization challenges . large language models (LLMs) can imitate personal writing styles, but they can generate fake news and misinformation. |
| Approach: | They propose a benchmark to evaluate detector robustness under personalization . they attribute this limitation to a feature-inversion trap that flips the effect in personalized contexts . |
| Outcome: | The proposed framework predicts detector robustness under personalization with an 85% correlation to actual results. |
Can LLMs Automate Fact-Checking Article Writing? (2026.tacl-1)
Copied to clipboard
Dhruv Sahnan, David Corney, Irene Larraz, Giovanni Zagni, Ruben Miguez, Zhuohan Xie, Iryna Gurevych, Elizabeth Churchill, Tanmoy Chakraborty, Preslav Nakov
| Challenge: | Existing tools for automatic fact-checking produce little or no justification for their assessments . 80% of american adults on major social media platforms regularly encounter news-related content . |
| Approach: | They propose to extend automatic fact-checking pipeline with automatic generation of full fact- checking articles. |
| Outcome: | The proposed framework outperforms existing frameworks but lags behind expert-written articles. |
Faithfulness-Aware Uncertainty Quantification for Fact-Checking the Output of Retrieval-Augmented Generation (2026.findings-acl)
Copied to clipboard
Ekaterina Fadeeva, Aleksandr Rubashevskii, Dzianis Piatrashyn, Roman Vashurin, Shehzaad Dhuliawala, Artem Shelmanov, Timothy Baldwin, Preslav Nakov, Mrinmaya Sachan, Maxim Panov
| Challenge: | Existing approaches to mitigating hallucinations conflate factuality with faithfulness to the retrieved evidence, incorrectly labeling factually correct statements as hallucinos . Existing methods to mitigate hallucinics rely on a lack of training data coverage, input ambiguity, and architectural constraints. |
| Approach: | They propose a method for hallucination detection in Large Language Models enhanced with knowledge retrieval based on faithfulness to the retrieved context. |
| Outcome: | The proposed method outperforms unsupervised UQ baselines, RAG-specific methods, and supervised classifiers across multiple tasks and LLMs. |
Beyond Memorization: Extending Reasoning Depth with Recurrence, Memory and Test-Time Compute Scaling (2026.findings-acl)
Copied to clipboard
Ivan Rodkin, Daniil Orel, Konstantin Smirnov, Arman Bolatov, Bilal Elbouardi, Besher Hassan, Yuri Kuratov, Aydar Bulatov, Preslav Nakov, Timothy Baldwin, Artem Shelmanov, Mikhail Burtsev
| Challenge: | Reasoning is a core capability of large language models, yet how multi-step reasoning is learned and executed remains unclear. |
| Approach: | They evaluate how large language models learn multi-step reasoning without memorization . they find that most neural architectures trained from scratch can learn rule inference . |
| Outcome: | The proposed framework fails to solve a natural-language proxy task with high accuracy. |
Multi-Task Ordinal Regression for Jointly Predicting the Trustworthiness and the Leading Political Ideology of News Media (N19-1)
Copied to clipboard
| Challenge: | a number of fact-checking initiatives have been launched, both manual and automatic, but the whole enterprise remains in a state of crisis. |
| Approach: | They propose a multi-task ordinal regression framework that models trustworthiness estimation and political ideology detection of entire news outlets. |
| Outcome: | The proposed model outperforms models that target the problems in isolation. |
Explicit and Implicit Data Augmentation for Social Event Detection (2025.acl-long)
Copied to clipboard
| Challenge: | Social event detection relies on labeled data, but annotation is costly and labor-intensive. |
| Approach: | They propose a plug-and-play dual augmentation framework that combines explicit text-based and implicit feature-space augmentation to enhance data diversity and model robustness. |
| Outcome: | The proposed framework outperforms the best baseline model by 17.67% on the Twitter2012 dataset and 15.57% on the twitter2018 dataset in terms of the average F1 score. |
VSCBench: Bridging the Gap in Vision-Language Model Safety Calibration (2025.findings-acl)
Copied to clipboard
Jiahui Geng, Qing Li, Zongxiong Chen, Yuxia Wang, Derui Zhu, Zhuohan Xie, Chenyang Lyu, Xiuying Chen, Preslav Nakov, Fakhri Karray
| Challenge: | Existing safety calibration methods focus on model undersafety, where the model responds to hazardous queries, while neglecting oversafetiness, where models refuse to answer safe queries. |
| Approach: | They propose safety calibration which addresses both undersafety and oversafetiness by comparing model responses to a novel dataset of 3,600 image-text pairs. |
| Outcome: | The proposed methods have been used to evaluate safety calibration across image-centric and text-centric scenarios. |
Batch-Softmax Contrastive Loss for Pairwise Sentence Scoring Tasks (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent advances in machine learning have led to the use of contrastive loss for representation learning. |
| Approach: | They propose to use batch-softmax contrastive loss to train pairwise sentence embeddings . they propose to take a batch-softermax contrastitive loss and train it with different loss functions . |
| Outcome: | The proposed model improves on a number of datasets and pairwise sentence scoring tasks. |
Uncovering Temporal Framing in the News (2026.acl-long)
Copied to clipboard
Tarek Mahmoud, Veronika Solopova, Premtim Sahitaj, Ariana Sahitaj, Max Upravitelev, Mervat Abassy, Hana Fatima Shaikh, Neda Foroutan, Vera Schmitt, Preslav Nakov
| Challenge: | Temporal language is used to structure meaning rather than report chronology in news discourse . a recent study focused on temporal expression extraction and temporal reasoning . |
| Approach: | They propose a taxonomy of eight temporal frames grounded in prior work on time and framing . they analyze frame prevalence, co-occurrence patterns, and lexical cues from a news corpus . |
| Outcome: | The proposed taxonomy outperforms zero-shot models at the sentence level . it shows that temporal framing is learnable at the sentences level compared to other methods . |
Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Understanding how jailbreaking works remains limited, hindering the development of effective defense strategies. |
| Approach: | They propose a new mechanism that adaptively constrains activations within the safety boundary and propose 'Activation Boundary Defense' to enhance its effectiveness. |
| Outcome: | The proposed defense achieves an average Defense Success Rate (DSR) of over 98% against various jailbreak attacks, with less than 2% impact on the model’s general capabilities. |
What Was Written vs. Who Read It: News Media Profiling Using Text Analysis and Social Media Context (2020.acl-main)
Copied to clipboard
Ramy Baly, Georgi Karadzhov, Jisun An, Haewoon Kwak, Yoan Dinkov, Ahmed Ali, James Glass, Preslav Nakov
| Challenge: | a growing number of fake news reports are published online, causing a trust crisis . a new study aims to predict political bias and factuality of reporting of entire news outlets . |
| Approach: | They propose to profile entire news outlets and look for those that are likely to publish fake content . they also examine what was written about the target medium and who reads it . |
| Outcome: | The proposed method improves on the current state-of-the-art in analyzing social media and what was written about the target medium. |
From Multiple-Choice to Extractive QA: A Case Study for English and Arabic (2025.coling-main)
Copied to clipboard
Teresa Lynn, Malik H. Altakrori, Samar M. Magdy, Rocktim Jyoti Das, Chenyang Lyu, Mohamed Nasr, Younes Samih, Kirill Chirkunov, Alham Fikri Aji, Preslav Nakov, Shantanu Godbole, Salim Roukos, Radu Florian, Nizar Habash
| Challenge: | Recent years have brought about very fast developments in Natural Language Processing (NLP), but many other languages are overlooked due to limited resources. |
| Approach: | They propose to repurpose a multilingual BELEBELE dataset for a task of extractive QA in the style of machine reading comprehension. |
| Outcome: | The proposed approach could be used to extract QA in the style of machine reading comprehension. |
That is a Known Lie: Detecting Previously Fact-Checked Claims (2020.acl-main)
Copied to clipboard
| Challenge: | a large number of fact-checked claims have been accumulated over the years . despite the importance of fact checking, it has been largely ignored by the research community . |
| Approach: | They propose to automate fact-checking by focusing on claims that have already been fact-tested . they propose to use specialized datasets to compare different methods . |
| Outcome: | The proposed task shows that it improves over state-of-the-art methods. |
Detecting Propaganda Techniques in Code-Switched Social Media Text (2023.emnlp-main)
Copied to clipboard
| Challenge: | a new study aims to detect propaganda in multiple languages using code-switching . social media platforms have made it easier for anyone to spread information to a wide audience . |
| Approach: | They propose to detect propaganda techniques in code-switched texts using a corpus of 1,030 texts . they propose to model multilinguality directly rather than using translation . |
| Outcome: | The proposed method combines different languages within the same text, presenting a challenge for automatic systems. |
A Survey on Predicting the Factuality and the Bias of News Media (2024.findings-acl)
Copied to clipboard
| Challenge: | a growing number of scholars are profiling entire news outlets to profile fake content . political bias detection is also an important topic, but the two problems have been addressed separately . |
| Approach: | They argue that media profiling should be based on factuality and bias together . they argue that it is difficult to fact-check every single suspicious claim or article manually . |
| Outcome: | The present level of proliferation of fake, biased, and propagandistic content online has made it impossible to fact-check every single suspicious claim or article, either manually or automatically. |
Grounding Fallacies Misrepresenting Scientific Publications in Evidence (2025.naacl-long)
Copied to clipboard
| Challenge: | Current fact-checking models struggle to use misrepresented scientific passages to refute misinformation. |
| Approach: | They propose an extension of the fallacy detection dataset Missci to detect and highlight such fallacies by grounding applied fallacias in real-world passages from misrepresented studies. |
| Outcome: | The proposed model can detect and highlight misrepresented publications and verbalize them to refute them. |
EXAMS-V: A Multi-Discipline Multilingual Multimodal Exam Benchmark for Evaluating Vision Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for vision language models are outdated and unable to accurately assess their performance. |
| Approach: | They propose a multi-discipline multimodal multilingual exam benchmark for vision language models . they collect multiple-choice questions across 20 disciplines across 11 languages from 7 language families . |
| Outcome: | The EXAMS-V exam includes 20,932 multiple-choice questions across 20 disciplines . the questions come in 11 languages from 7 language families and require advanced reasoning skills . |
SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables (2023.emnlp-main)
Copied to clipboard
| Challenge: | Current scientific fact-checking benchmarks exhibit several shortcomings, such as biases arising from crowd-sourced claims and an over-reliance on text-based evidence. |
| Approach: | They present a dataset of 1.2K expert-verified scientific claims that require compositional reasoning for verification. |
| Outcome: | The proposed model outperforms existing models in table-based pretraining models and large language models. |
Findings of the NLP4IF-2019 Shared Task on Fine-Grained Propaganda Detection (D19-50)
Copied to clipboard
| Challenge: | A shared task on fine-grained propaganda detection was organized at EMNLP-IJCNLP 2019 . 12 systems submitted systems for the FLC task, 25 for the SLC task, and 14 teams submitted a system description paper . |
| Approach: | They present a task on fine-grained propaganda detection as part of the NLP4IF workshop at EMNLP-IJCNLP 2019 . they used a corpus of news articles annotated with an inventory of propagandist techniques at the fragment level to determine the propaganda technique used in each fragment . |
| Outcome: | The shared task on fine-grained propaganda detection was organized at the EMNLP-IJCNLP 2019 . 12 systems submitted for the FLC task, 25 for the SLC task, and 14 submitted a system description paper . |
Large Language Models are Few-Shot Training Example Generators: A Case Study in Fallacy Recognition (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing work on fallacy recognition is still in its early stages, with limited datasets available. |
| Approach: | They propose to use GPT3.5 to generate synthetic examples and explore prompt settings to improve the representation of the infrequent classes. |
| Outcome: | The proposed model improves on existing models and generates synthetic examples with GPT3.5. |
Nanda Family: Open-Weights Generative Large Language Models for Hindi (2026.eacl-long)
Copied to clipboard
Aaryamonvikram Singh, Debopriyo Banerjee, Dhruv Sahnan, Monojit Choudhury, Shivam Chauhan, Rocktim Jyoti Das, Xudong Han, Haonan Li, Alok Anil Jadhav, Utkarsh Agarwal, Mukund Choudhary, Fajri Koto, Junaid Hamid Bhat, Awantika Shukla, Samujjwal Ghosh, Samta Kamboj, Onkar Pandit, Lalit Pradhan, Rahul Pal, Sunil Kumar Sahu, Parvez Mullah, Ali El Filali, Zainul Abedien Ahmed Quraishi, Neha Sengupta, Gokulakrishnan Ramakrishnan, Rituraj Joshi, Gurpreet Gosal, Avraham Sheinin, Natalia Vassilieva, Preslav Nakov
| Challenge: | Large language models remain predominantly English-centric, which limits their utility for underrepresented languages. |
| Approach: | They propose to extend Llama’s vocabulary with 20% Hindi-specific tokens, thus halving Hindi tokenization fertility while preserving English efficiency. |
| Outcome: | The proposed models outperform open-weight models of comparable size on a 65B-token corpus and bilingual instruction and safety alignment on . a culturally grounded dataset. |
Fact-Checking, Fake News, Propaganda, and Media Bias: Truth Seeking in the Post-Truth Era (2020.emnlp-tutorials)
Copied to clipboard
| Challenge: | social media has made it easy for everyone to share and spread information online. |
| Approach: | a tutorial will offer an overview of the broad and emerging research area of disinformation . it will focus on the latest developments and research directions . |
| Outcome: | The tutorial will offer an overview of the broad and emerging research area of disinformation . it will focus on the latest developments and research directions . |
Beyond the Crowd: LLM-Augmented Community Notes for Governing Health Misinformation (2026.acl-long)
Copied to clipboard
| Challenge: | X (formerly Twitter) users can flag misleading posts, attach contextual notes, and rate the notes’ helpfulness, but there is a significant latency in Community Notes, which is unable to provide accurate notes. |
| Approach: | They propose a framework that augments Community Notes for faster and more reliable health misinformation governance. |
| Outcome: | The proposed framework outperforms human contributors in correctness, helpfulness, and evidence utility in health misinformation surges. |
Factuality of Large Language Models: A Survey (2024.emnlp-main)
Copied to clipboard
Yuxia Wang, Minghan Wang, Muhammad Arslan Manzoor, Fei Liu, Georgi Georgiev, Rocktim Das, Preslav Nakov
| Challenge: | Large language models (LLMs) are factually incorrect, which limits their applicability in real-world scenarios. |
| Approach: | They analyze existing work to identify major challenges and their associated causes . they propose to evaluate LLMs using a variety of measures to mitigate factual errors . |
| Outcome: | The proposed methods are based on a variety of datasets and proposed strategies to mitigate factual errors. |
GREENER: Graph Neural Networks for News Media Profiling (2022.emnlp-main)
Copied to clipboard
| Challenge: | a new method for profiling news media on the Web addresses the factuality of reporting and bias problem . a recent study has focused on text features but has focused primarily on text . |
| Approach: | They propose a model that models the similarity between media outlets based on their audience overlap . they propose GREENER, which builds a graph of inter-media connections based upon audience overlap. |
| Outcome: | The proposed model improves on state-of-the-art models on two datasets. |
Fighting the COVID-19 Infodemic: Modeling the Perspective of Journalists, Fact-Checkers, Social Media Platforms, Policy Makers, and the Society (2021.findings-emnlp)
Copied to clipboard
Firoj Alam, Shaden Shaar, Fahim Dalvi, Hassan Sajjad, Alex Nikolov, Hamdy Mubarak, Giovanni Da San Martino, Ahmed Abdelali, Nadir Durrani, Kareem Darwish, Abdulaziz Al-Homaid, Wajdi Zaghouani, Tommaso Caselli, Gijs Danoe, Friso Stolk, Britt Bruntink, Preslav Nakov
| Challenge: | a dataset of 16K manually annotated tweets is used to analyze disinformation . the democratic nature of social media has raised questions about the quality and the factuality of the information that is shared on these platforms. |
| Approach: | They use a dataset of manually annotated tweets to analyze COVID-19 disinformation . they show that tweets contain fake cures, rumors, conspiracy theories and xenophobia . |
| Outcome: | The proposed dataset shows that it is useful in monolingual vs. multilingual settings. |
ClaimRank: Detecting Check-Worthy Claims in Arabic and English (N18-5)
Copied to clipboard
| Challenge: | ClaimRank is an online system for detecting check-worthy claims . it can be used to prioritize the claims fact-checkers should consider first . |
| Approach: | ClaimRank is an online system for detecting check-worthy claims . it is originally trained on political debates, but can work for any kind of text . authors propose to make automated fact-checking easier by prioritizing claims based on annotations from reputable fact- checking organizations. |
| Outcome: | ClaimRank is an online system for detecting check-worthy claims . it can mimic the sentence selection strategies of reputable fact-checking organizations . |
Corpus Poisoning via Approximate Greedy Gradient Descent (2025.findings-acl)
Copied to clipboard
| Challenge: | Dense retrievers have been shown to be vulnerable to corpus poisoning attacks . a malicious user injects a small fraction of adversarial passages into the retrieval corpus to trick the system into returning these passages among the top-ranked results for a broad set of user queries. |
| Approach: | They propose an attack on dense retrieval systems based on the widely used HotFlip method for efficiently generating adversarial passages. |
| Outcome: | The proposed method achieves a high success rate on several datasets and using several retrievers, and can generalize to unseen queries and new domains. |
DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models generate huge amounts of text, making it impractical to manually distinguish whether a text is machine-generated. |
| Approach: | They propose two methods to detect machine-generated text by leveraging Log-Rank information and propose a faster method that uses less perturbations to achieve the same level of performance. |
| Outcome: | The proposed methods improve over the state of the art by 3.9 and 1.75 AUROC points absolute and require less perturbations to achieve the same level of performance. |
Health-ORSC-Bench: A Benchmark for Measuring Over-Refusal and Safety Completion in Health Context (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing safety alignment benchmarks fail to evaluate Safe Completion: the model’s ability to maximise helpfulness on dual-use or borderline queries without crossing into actionable harm. |
| Approach: | They propose a large-scale benchmark to measure Over-Refusal and Safe Completion quality in healthcare. |
| Outcome: | The framework evaluates 30 state-of-the-art LLMs including GPT-5 and Claude-4. |
Detecting Harmful Memes and Their Targets (2021.findings-acl)
Copied to clipboard
Shraman Pramanick, Dimitar Dimitrov, Rituparna Mukherjee, Shivam Sharma, Md. Shad Akhtar, Preslav Nakov, Tanmoy Chakraborty
| Challenge: | a growing body of research on meme analysis has focused on detecting harmful memes and their social entities . a meme is a form of content that is often harmless and designed to look funny . but its multimodal nature and camouflaged semantics make its analysis challenging . |
| Approach: | They propose to use multimodal models to detect harmful memes and identify social entities that harmful meme targets. |
| Outcome: | The proposed model can detect harmful memes and the social entities they target . the proposed model lacks the appropriate contexts and is poorly validated . |
Faking Fake News for Real Fake News Detection: Propaganda-Loaded Training Data Generation (2023.acl-long)
Copied to clipboard
| Challenge: | despite advances in detecting fake news, there is a sizable gap between machine-generated and human-authored fake news . a nave solution is to collect human-written news articles that contain inaccurate information by crawling untrustworthy news media. |
| Approach: | They propose a framework for generating training examples informed by the styles and strategies of human-authored propaganda. |
| Outcome: | The proposed framework improves detection of human-written disinformation by 3.62–7.69% on two public datasets. |
Cultural Benchmarking of LLMs in Standard and Dialectal Arabic Dialogues (2026.acl-long)
Copied to clipboard
Muhammad Dehan Al Kautsar, Saeed Almheiri, Momina Ahsan, Bilal Elbouardi, Younes Samih, Sarfraz Ahmad, Amr Keleg, Omar El Herraoui, Kareem Elzeky, Abed Alhakim Freihat, Mohamed Anwar, Zhuohan Xie, Junhong Liang, Mohammad Rustom Al Nasar, Preslav Nakov, Fajri Koto
| Challenge: | Most benchmarks focus on short text snippets in Modern Standard Arabic (MSA), overlooking cultural nuances that naturally arise in dialogues. |
| Approach: | They propose a culturally grounded conversational dataset covering 13 Arabic-speaking countries, in both Modern Standard Arabic (MSA) and each country’s respective dialect, spanning 12 daily-life topics and 54 fine-grained subtopics. |
| Outcome: | The proposed model performs worse on all three tasks than the MSA benchmark. |
MemeMQA: Multimodal Question Answering for Memes via Rationale-Based Inferencing (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on harms of memes in closed environments, such as hate speech and cyber-bullying. |
| Approach: | They propose a multimodal question-answering framework that solicits accurate responses to structured questions while providing coherent explanations. |
| Outcome: | The proposed framework outperforms existing frameworks in predicting answer prediction accuracy and text generation lead over a baseline. |
Automatic Stance Detection Using End-to-End Memory Networks (N18-1)
Copied to clipboard
| Challenge: | Existing methods for fact checking are tedious and often broken into intermediate steps to alleviate complexity. |
| Approach: | They propose an end-to-end memory network model that predicts whether a document can be considered relevant for a given claim and extracts relevant text snippets. |
| Outcome: | The proposed model predicts whether a document can be considered relevant for a given claim and extracts relevant text snippets to reason about the factuality of the target claim. |
Recent Advances in Online Hate Speech Moderation: Multimodality and the Role of Large Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | HS is any communication demeaning a person or a group based on social or ethnic characteristics that undermines social harmony and individual safety . the recent Israel-Hamas conflict has escalated both anti-Muslim and anti-Semitic sentiments worldwide . |
| Approach: | They examine the role of large language models and large multimodal models in HS moderation . they examine how text, images, and audio interact to spread hate speech . |
| Outcome: | The findings highlight the need for solutions in low-resource settings and highlight the gaps in existing methods. |