Challenge: a large-scale dataset of 186K articles from 1.5K newspapers published in the summer of 2025 is audited.
Approach: They audit 186K articles from 1.5K newspapers published in summer of 2025 . they use Pangram, a state-of-the-art AI detector, to detect whether articles are partially or fully AI-generated .
Outcome: The findings highlight the need for greater transparency and updated editorial standards regarding the use of AI in journalism to maintain public trust.

Similar Papers

Adapting Fake News Detection to the Era of Large Language Models (2024.findings-naacl)

Copied to clipboard

Challenge: a gap exists in understanding the interplay between machine-paraphrased real news, machine-generated fake news, and human-written real news . false information is easier to generate but harder to detect due to the bias of detectors against machine-generated texts .
Approach: They propose a strategy to adapt fake news detectors to the era of large language models and AI-driven content creation .
Outcome: The proposed detectors perform well on human-written articles but not vice versa . the proposed detector should be trained on datasets with lower machine-generated news ratio than the test set .
Human Bias in the Face of AI: Examining Human Judgment Against Text Labeled as AI Generated (2025.findings-acl)

Copied to clipboard

Challenge: Prior research on AI mistrust focused primarily on AI's bias towards different human pop-ups.
Approach: They examine how bias shapes the perception of AI versus human generated content . they found that raters favored content labeled "Human Generated" even when labels were deliberately swapped .
Outcome: The findings highlight the limitations of human judgment in interacting with AI and offer a foundation for improving human-AI collaboration.
A Practical Examination of AI-Generated Text Detectors for Large Language Models (2025.findings-naacl)

Copied to clipboard

Challenge: Existing methods to detect large language models are prone to misuse, such as generating fake news articles, facilitating academic plagiarism or spamming.
Approach: They evaluate several popular detectors to evaluate their effectiveness against a range of domains, datasets, and models.
Outcome: The proposed methods perform poorly in certain settings, with TPR@.01 as low as 0%.
Discovering Biased News Articles Leveraging Multiple Human Annotations (2020.lrec-1)

Copied to clipboard

Challenge: Political propaganda and one-sided views can be found in the news and can cause distrust in media.
Approach: They propose to annotate politically biased news articles by an algorithm annotated by domain experts and crowd workers and to compare them to crowd workers.
Outcome: The proposed method compares domain experts to crowd workers and shows that bias can be detected automatically.
All That Glitters is Not Novel: Plagiarism in AI Generated Research (2025.acl-long)

Copied to clipboard

Challenge: Recent studies claim autonomous research agents can generate novel research ideas.
Approach: They ask experts to evaluate whether existing work is similar to new ones . they find 24% of the 50 evaluated documents to be either paraphrased or significantly borrowed .
Outcome: The authors find that 24% of the 50 evaluated research documents are either paraphrased, or significantly borrowed from existing work.
Invisible to People but not to Machines: Evaluation of Style-aware HeadlineGeneration in Absence of Reliable Human Judgment (2020.lrec-1)

Copied to clipboard

Challenge: Using a data alignment strategy and different training/testing settings, we aim at decoupling content from style and preserving the latter in generation.
Approach: They propose a fine-grained evaluation strategy based on automatic classification to evaluate generated headlines' quality in terms of their newspaper-compliance.
Outcome: The proposed model learns newspaper-specific style, but humans aren't reliable judges for this task, and deserves particular care in its design.
Have LLMs Reopened the Pandora’s Box of AI-Generated Fake News? (2025.naacl-long)

Copied to clipboard

Challenge: Large language models (LLMs) are increasingly being used by fake news creators to generate deceptive and persuasive content at scale.
Approach: They propose to use large language models to generate fake news at scale and to assess the ability of human annotators and AI models to detect it.
Outcome: The results show that LLMs are 68% more effective at detecting real news than humans, compared to humans and AI models for fake news detection.
A Case Study on Neural Headline Generation for Editing Support (N19-2)

Copied to clipboard

Challenge: a news-aggregator is a website or mobile application that aggregates web content . dozens of professional editors manually create their headlines, which are much shorter than the original headlines.
Approach: They propose a neural headline generation model that automatically generates short headlines from news articles.
Outcome: The proposed model is deployed to an editing support tool and compares editors' behavior before and after the release.
Almost AI, Almost Human: The Challenge of Detecting AI-Polished Writing (2025.findings-acl)

Copied to clipboard

Challenge: a growing use of large language models (LLMs) has led to concerns about AI-generated content detection.
Approach: They evaluate 12 state-of-the-art AI-text detectors using a dataset refined at varying levels of AI involvement.
Outcome: The proposed detectors flag even minimally polished text as AI-generated, struggle to differentiate between degrees of AI involvement, and exhibit biases against older and smaller models.
Exploring the Limitations of Detecting Machine-Generated Text (2025.coling-main)

Copied to clipboard

Challenge: Recent advances in the quality of the generation of text by large language models have spurred research into identifying machine-generated text.
Approach: They audit classification performance for detecting machine-generated text by evaluating on texts with varying writing styles.
Outcome: The proposed methods are highly sensitive to stylistic changes and complexity, and in some cases degrade entirely to random classifiers.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations