CoDet-M4: Detecting Machine-Generated Code in Multi-Lingual, Multi-Generator and Multi-Domain Settings (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have revolutionized code generation but have significant consequences for programming skills, ethics, and assessment integrity. |
| Approach: | They propose a framework capable of distinguishing between human-written and LLM-generated program code across multiple programming languages, code generators, and domains. |
| Outcome: | The proposed framework distinguishes between human-written and LLM-generated program code across multiple programming languages, code generators, and domains. |
Similar Papers
M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection (2024.eacl-long)
Copied to clipboard
Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Chenxi Whitehouse, Osama Mohammed Afzal, Tarek Mahmoud, Toru Sasaki, Thomas Arnold, Alham Aji, Nizar Habash, Iryna Gurevych, Preslav Nakov
| Challenge: | Large language models generate fluent responses to user queries, but they are also susceptible to misuse in journalism, education, and academia. |
| Approach: | They propose a large-scale benchmark for machine-generated text detection that is a multi-generator, multi-domain, and multi-lingual corpus. |
| Outcome: | The proposed system can detect machine-generated text and pinpoint misuse . the proposed system is based on a large-scale benchmark dataset . |
Representation and Generation of Machine Learning Test Functions (2024.eacl-srw)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been adopted for ML code generation but their implications are relatively unexplored. |
| Approach: | They examine the use of Large Language Models to extract representations of ML source code and tests to understand the semantic relationships between human-written tests and LLM-generated ones. |
| Outcome: | The proposed models can be used to extract representations of ML source code and tests and annotate them for usefulness, documentation, and correctness. |
M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection (2024.acl-long)
Copied to clipboard
Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Osama Mohammed Afzal, Tarek Mahmoud, Giovanni Puccetti, Thomas Arnold, Alham Aji, Nizar Habash, Iryna Gurevych, Preslav Nakov
| Challenge: | Large Language Models (LLMs) have brought an unprecedented surge in machine-generated text (MGT) societal implications are posed by their potential misuse and lack of training data. |
| Approach: | They propose a benchmark to detect machine-generated text in multiple languages . they use multi-domain and multi-generator corpus to identify which model generated the text . |
| Outcome: | The proposed benchmark compares a multilingual, multi-domain and multi-generator corpus of MGTs with human-generated content. |
LLM-DetectAIve: a Tool for Fine-Grained Machine-Generated Text Detection (2024.emnlp-demo)
Copied to clipboard
Mervat Abassy, Kareem Elozeiri, Alexander Aziz, Minh Ta, Raj Tomar, Bimarsha Adhikari, Saad Ahmed, Yuxia Wang, Osama Mohammed Afzal, Zhuohan Xie, Jonibek Mansurov, Ekaterina Artemova, Vladislav Mikhailov, Rui Xing, Jiahui Geng, Hasan Iqbal, Zain Mujahid, Tarek Mahmoud, Akim Tsvigun, Alham Aji, Artem Shelmanov, Nizar Habash, Iryna Gurevych, Preslav Nakov
| Challenge: | a large number of machine-generated texts are often hard to distinguish between human-written and machine-generated text . this raises concerns about potential misuse, especially within educational and academic domains . |
| Approach: | They propose a system that can detect whether a text is human-written or machine-generated . they use a fine-grained classification schema to identify the use of machine-generated text . |
| Outcome: | The proposed system can distinguish between human-written and machine-generated text . it can detect attempts to obfuscate the fact that a text was machine- generated . |
MAGE: Machine-generated Text Detection in the Wild (2024.acl-long)
Copied to clipboard
Yafu Li, Qintong Li, Leyang Cui, Wei Bi, Zhilin Wang, Longyue Wang, Linyi Yang, Shuming Shi, Yue Zhang
| Challenge: | Existing research has focused on evaluating detection methods for specific domains or language models. |
| Approach: | They build a testbed to detect texts from diverse human writings and LLMs using different detection methods. |
| Outcome: | Empirical results show that the top performing detector can identify 84.12% out-of-domain texts generated by a new LLM, indicating the feasibility for application scenarios. |
Linguistic and Embedding-Based Profiling of Texts Generated by Humans and Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have focused on using LLMs to classify text as either human-written or machine-generated . |
| Approach: | They characterize human-written and machine-generated texts using a set of linguistic features across different linguistic levels such as morphology, syntax, and semantics. |
| Outcome: | The proposed model reveals that human-written texts exhibit simpler syntactic structures and more diverse semantic content. |
CodeIP: A Grammar-Guided Multi-Bit Watermark for Large Language Models of Code (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown significant potential in code generation, but they also present challenges regarding the protection of Intellectual Property (IP) related to model architectures, weights, and training data. |
| Approach: | They propose a multi-bit watermarking technique that embeds additional information to preserve provenance details, such as the vendor ID of an LLM. |
| Outcome: | The proposed technique preserves provenance details while maintaining syntactical correctness of generated code. |
PlagBench: Exploring the Duality of Large Language Models in Plagiarism Generation and Detection (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent studies have raised concerns about the potential threats large language models pose to academic integrity and copyright protection. |
| Approach: | They propose a dataset of 46.5K synthetic text pairs that represent three major types of plagiarism: verbatim copying, paraphrasing, and summarization. |
| Outcome: | The proposed dataset shows that GPT-3.5 Turbo can produce high-quality paraphrases and summaries without significantly increasing text complexity compared to GPT-4 Turbo. |
Authorship Attribution in Multilingual Machine-Generated Texts (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have reached human-like fluency and coherence, but distinguishing machine-generated text from human-written content becomes increasingly difficult. |
| Approach: | They propose a problem of multilingual authorship attribution (AA) that involves attributing texts to human or multiple LLM generators across diverse languages. |
| Outcome: | The proposed method can be adapted to multilingual settings, but still has significant limitations and challenges. |
LLM-as-a-Coauthor: Can Mixed Human-Written and Machine-Generated Text Be Detected? (2024.findings-naacl)
Copied to clipboard
Qihui Zhang, Chujie Gao, Dongping Chen, Yue Huang, Yixin Huang, Zhenyang Sun, Shilin Zhang, Weiye Li, Zhengyan Fu, Yao Wan, Lichao Sun
| Challenge: | Current research focuses on purely MGT detection without adequately addressing mixed scenarios including AI-revised Human-Written Text (HWT) and human-revealed MGT. |
| Approach: | They define mixtext, a form of mixed text involving both AI and human-generated content, and then use a MixSet dataset to assess their effectiveness. |
| Outcome: | The proposed detectors struggle to identify mixtext, particularly in dealing with subtle modifications and style adaptability. |