Counter Turing Test (CT2): AI-Generated Text Detection is Not as Easy as You May Think - Introducing AI Detectability Index (ADI) (2023.emnlp-main)
Copied to clipboard
Megha Chakraborty, S.M Towhidul Islam Tonmoy, S M Mehedi Zaman, Shreya Gautam, Tanay Kumar, Krish Sharma, Niyar Barman, Chandan Gupta, Vinija Jain, Aman Chadha, Amit Sheth, Amitava Das
| Challenge: | a number of issues have arisen regarding the risk and consequences of AI-generated text detection. |
| Approach: | They propose a counter-turing test to evaluate the robustness of existing AGTD methods . they propose ADI, a quantifiable spectrum to assess detectability of LLMs . |
| Outcome: | The proposed method evaluates the robustness of existing AGTD methods . it shows that larger LLMs tend to have lower ADI, indicating they are less detectable . |
Similar Papers
Counter Turing Test (CT2): Investigating AI-Generated Text Detection for Hindi - Ranking LLMs based on Hindi AI Detectability Index (ADI_hi) (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a growing number of large language models are being used to detect AI-generated text . a recent study has found that some techniques to bypass detection are fragile . |
| Approach: | They propose to use 26 LLMs to evaluate their proficiency in generating Hindi text . they propose to introduce a Hindi AI Detectability Index to assess and rank LLM models based on their detectability levels. |
| Outcome: | The proposed methods are effective in English, but struggle in Hindi . the proposed methods show that they are susceptible to fragility . |
TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in generative language models have enabled machines to generate realistic texts. |
| Approach: | They propose a benchmark environment to test the 'Turing Test' problem for neural text generation methods. |
| Outcome: | The proposed benchmark environment is based on 200K human- or machine-generated samples across 20 labels Human, GPT-1, GTP-2_small, GTT-2_medium, GPG-2_large, GGT-2_PyTorch, GGP-3, GROVER_base, griover_large and GRover_mega. |
People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text (2025.acl-long)
Copied to clipboard
| Challenge: | Qualitative analysis of experts’ free-form explanations shows that while they rely heavily on specific lexical clues (‘AI vocabulary’), they also pick up on more complex phenomena within the text (e.g., formality, originality, clarity). |
| Approach: | They hire annotators to read 300 non-fiction English articles, label them as either human-written or AI-generated, and provide paragraph-length explanations for their decisions. |
| Outcome: | The annotators who frequently use LLMs for writing tasks outperform commercial and open-source detectors even without evasion tactics like paraphrasing and humanization. |
LLM-DetectAIve: a Tool for Fine-Grained Machine-Generated Text Detection (2024.emnlp-demo)
Copied to clipboard
Mervat Abassy, Kareem Elozeiri, Alexander Aziz, Minh Ta, Raj Tomar, Bimarsha Adhikari, Saad Ahmed, Yuxia Wang, Osama Mohammed Afzal, Zhuohan Xie, Jonibek Mansurov, Ekaterina Artemova, Vladislav Mikhailov, Rui Xing, Jiahui Geng, Hasan Iqbal, Zain Mujahid, Tarek Mahmoud, Akim Tsvigun, Alham Aji, Artem Shelmanov, Nizar Habash, Iryna Gurevych, Preslav Nakov
| Challenge: | a large number of machine-generated texts are often hard to distinguish between human-written and machine-generated text . this raises concerns about potential misuse, especially within educational and academic domains . |
| Approach: | They propose a system that can detect whether a text is human-written or machine-generated . they use a fine-grained classification schema to identify the use of machine-generated text . |
| Outcome: | The proposed system can distinguish between human-written and machine-generated text . it can detect attempts to obfuscate the fact that a text was machine- generated . |
Learning to Rewrite: Generalized LLM-Generated Text Detection (2025.acl-long)
Copied to clipboard
| Challenge: | Existing detectors for Large Language Models (LLMs) struggle to generalize in open-world settings. |
| Approach: | They propose a framework to detect LLM-generated text with exceptional generalization to unseen domains by reinforcing LLMs’ inherent rewriting tendencies. |
| Outcome: | The proposed framework outperforms state-of-the-art detection methods by 23.04% in AUROC, 35.10% for out-of distribution tests, and 48.66% under adversarial attacks. |
A Survey on Detection of LLMs-Generated Content (2024.findings-emnlp)
Copied to clipboard
Xianjun Yang, Liangming Pan, Xuandong Zhao, Haifeng Chen, Linda Petzold, William Yang Wang, Wei Cheng
| Challenge: | Recent advances in large language models have led to an increase in synthetic content generation . the ability to detect LLMs-generated content has become of paramount importance . |
| Approach: | They propose to provide a detailed overview of existing detection strategies and benchmarks, scrutinizing their differences and advocating for more adaptable and robust models to enhance detection accuracy. |
| Outcome: | The proposed model will be able to detect human-written content in real time. |
FAID: Fine-grained AI-generated Text Detection using Multi-task Auxiliary and Multi-level Contrastive Learning (2026.eacl-long)
Copied to clipboard
Minh Ngoc Ta, Dong Cao Van, Duc-Anh Hoang, Minh Le-Anh, Truong Nguyen, My Anh Tran Nguyen, Yuxia Wang, Preslav Nakov, Dinh Viet Sang
| Challenge: | Existing binary detection frameworks for human-written, LLM-generated and human-LLM collaborative texts are challenging . a recent study focused on binary detection, i.e., human vs. LLM, or on fine-grained detection limited to English. |
| Approach: | They propose a fine-grained detection framework to classify text into three categories . they use multilingual datasets and a multi-domain, multi-generator dataset . |
| Outcome: | The proposed framework outperforms baselines on unseen domains and new LLMs. |
MOSAIC: Multiple Observers Spotting AI Content (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have made it easier for all to produce harmful, toxic, faked or forged content. |
| Approach: | They propose to use large language models to automatically discriminate from human-written texts by comparing their probability distributions over a document to see if they can detect forged or harmful content. |
| Outcome: | The proposed approach harnesses each model’s capabilities, leading to strong detection performance on a variety of domains. |
Almost AI, Almost Human: The Challenge of Detecting AI-Polished Writing (2025.findings-acl)
Copied to clipboard
| Challenge: | a growing use of large language models (LLMs) has led to concerns about AI-generated content detection. |
| Approach: | They evaluate 12 state-of-the-art AI-text detectors using a dataset refined at varying levels of AI involvement. |
| Outcome: | The proposed detectors flag even minimally polished text as AI-generated, struggle to differentiate between degrees of AI involvement, and exhibit biases against older and smaller models. |
PlagBench: Exploring the Duality of Large Language Models in Plagiarism Generation and Detection (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent studies have raised concerns about the potential threats large language models pose to academic integrity and copyright protection. |
| Approach: | They propose a dataset of 46.5K synthetic text pairs that represent three major types of plagiarism: verbatim copying, paraphrasing, and summarization. |
| Outcome: | The proposed dataset shows that GPT-3.5 Turbo can produce high-quality paraphrases and summaries without significantly increasing text complexity compared to GPT-4 Turbo. |