IMGTB: A Framework for Machine-Generated Text Detection Benchmarking (2024.acl-demos)
Copied to clipboard
| Challenge: | MGTD methods are needed in many areas, such as prevention of disinformation spreading, plagiarism, impersonation and identity theft. |
| Approach: | They propose a framework for machine-generated text detection that integrates custom methods and evaluation datasets into existing frameworks. |
| Outcome: | The proposed framework simplifies the benchmarking of machine-generated text detection methods by easy integration of custom (new) methods and evaluation datasets. |
Similar Papers
M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection (2024.acl-long)
Copied to clipboard
Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Osama Mohammed Afzal, Tarek Mahmoud, Giovanni Puccetti, Thomas Arnold, Alham Aji, Nizar Habash, Iryna Gurevych, Preslav Nakov
| Challenge: | Large Language Models (LLMs) have brought an unprecedented surge in machine-generated text (MGT) societal implications are posed by their potential misuse and lack of training data. |
| Approach: | They propose a benchmark to detect machine-generated text in multiple languages . they use multi-domain and multi-generator corpus to identify which model generated the text . |
| Outcome: | The proposed benchmark compares a multilingual, multi-domain and multi-generator corpus of MGTs with human-generated content. |
On the Zero-Shot Generalization of Machine-Generated Text Detectors (2023.findings-emnlp)
Copied to clipboard
| Challenge: | rampant proliferation of large language models generates text indistinguishable from human-written language. |
| Approach: | They train neural detectors on outputs of a new generator and test their performance on held-out generators. |
| Outcome: | The proposed detectors can be built on training data from medium-sized models. |
Exploring the Limitations of Detecting Machine-Generated Text (2025.coling-main)
Copied to clipboard
| Challenge: | Recent advances in the quality of the generation of text by large language models have spurred research into identifying machine-generated text. |
| Approach: | They audit classification performance for detecting machine-generated text by evaluating on texts with varying writing styles. |
| Outcome: | The proposed methods are highly sensitive to stylistic changes and complexity, and in some cases degrade entirely to random classifiers. |
A Practical Examination of AI-Generated Text Detectors for Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to detect large language models are prone to misuse, such as generating fake news articles, facilitating academic plagiarism or spamming. |
| Approach: | They evaluate several popular detectors to evaluate their effectiveness against a range of domains, datasets, and models. |
| Outcome: | The proposed methods perform poorly in certain settings, with TPR@.01 as low as 0%. |
DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models generate huge amounts of text, making it impractical to manually distinguish whether a text is machine-generated. |
| Approach: | They propose two methods to detect machine-generated text by leveraging Log-Rank information and propose a faster method that uses less perturbations to achieve the same level of performance. |
| Outcome: | The proposed methods improve over the state of the art by 3.9 and 1.75 AUROC points absolute and require less perturbations to achieve the same level of performance. |
Detecting Machine-Generated Text: Techniques and Challenges (2024.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial focuses on machine-generated text and deepfakes. |
| Approach: | This tutorial aims to provide a comprehensive overview of text detection techniques . it will focus on machine-generated text and deepfakes . |
| Outcome: | This tutorial focuses on machine-generated text and deepfakes. |
Ghostbuster: Detecting Text Ghostwritten by Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Ghostbuster is a system that passes documents through weaker language models, runs a structured search over possible combinations of their features, and trains a classifier on the selected features. |
| Approach: | They propose a method that passes documents through weaker language models, runs a structured search over possible combinations of their features, and trains a classifier on the selected features. |
| Outcome: | The proposed method outperforms existing detectors and a new baseline on student essays, creative writing, and news articles. |
EvoBench: Towards Real-world LLM-Generated Text Detection Benchmarking for Evolving Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to detect LLM-generated texts rely on static benchmarks that neglect the evolving nature of LLMs. |
| Approach: | They propose a benchmark to evaluate the generalization of LLM-generated text detection methods. |
| Outcome: | The proposed benchmark measures generalization of 14 detection methods across LLMs. |
Reliably Bounding False Positives: A Zero-Shot Machine-Generated Text Detection Framework via Multiscaled Conformal Prediction (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods focus excessively on detection accuracy, neglecting the societal risks posed by high false positive rates (FPRs). |
| Approach: | They propose a Conformal Prediction framework that constrains the upper bound of false positive rates and introduces a real-time detection framework. |
| Outcome: | The proposed framework reduces false positive rates and improves detection performance. |
OpenTuringBench: An Open-Model-based Benchmark and Framework for Machine-Generated Text Detection and Attribution (2025.emnlp-main)
Copied to clipboard
| Challenge: | Open Large Language Models (OLLMs) are increasingly leveraged in generative AI applications, posing new challenges for detecting their outputs. |
| Approach: | They propose a benchmark to train and evaluate machine-generated text detectors on Turing Test and Authorship Attribution problems. |
| Outcome: | The proposed detector outperforms existing detectors in varying degrees of difficulty and relevance across tasks. |