Human-in-the-Loop Synthetic Text Data Inspection with Provenance Tracking (2024.findings-naacl)
Copied to clipboard
| Challenge: | Data augmentation techniques generate low-quality texts with incorrect labels . a new technique is needed to winnow out texts with inaccurate labels based on provenance inspection . |
| Approach: | They develop a data inspection technique that uses provenance inspection and assistive labeling to winnow out texts with incorrect labels. |
| Outcome: | a new human-in-the-loop data inspection technique can winnow out texts with incorrect labels . the technique can reduce human inspection effort by combining provenance inspection and assistive labeling . |
Similar Papers
Don’t Augment, Rewrite? Assessing Abusive Language Detection with Synthetic Data (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets for abusive language detection and content moderation are limited by regulatory bodies and social media platforms. |
| Approach: | They propose to replace existing datasets in English with synthetic data by rewriting original texts with an instruction-based generative model. |
| Outcome: | The proposed model improves performance in cross-dataset training. |
Automatic Detection of Generated Text is Easiest when Humans are Fooled (2020.acl-main)
Copied to clipboard
| Challenge: | Recent advances in neural language modelling make it possible to rapidly generate vast amounts of human-sounding text. |
| Approach: | They compare decoding methods with popular sampling-based decoding strategies . they show that multi-sentence excerpts can fool expert human raters over 30% of the time . |
| Outcome: | The proposed methods improve with longer excerpt length, but multi-sentence excerpts fool human raters over 30% of the time. |
FIND: Human-in-the-Loop Debugging Deep Text Classifiers (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models are limited in the number of available datasets and lack the necessary tools to improve them. |
| Approach: | They propose a framework which enables humans to debug deep learning text classifiers by disabling irrelevant hidden features. |
| Outcome: | Experiments show that using FIND, humans can improve CNN text classifiers trained on different types of imperfect datasets. |
Unraveling the Mystery of Artifacts in Machine Generated Text (2022.lrec-1)
Copied to clipboard
| Challenge: | Recent studies show that human-written text is not distinguishable from synthetic text because of semantic errors or logical contradictions. |
| Approach: | They propose to analyze the forms of artifacts left by neural Text Generation Models by corrupting texts and replacing them with linguistic or statistical features. |
| Outcome: | The proposed method replaces text with linguistic or statistical features and improves the accuracy of the model. |
GLTR: Statistical Detection and Visualization of Generated Text (P19-3)
Copied to clipboard
| Challenge: | GLTR is a tool to detect generated text that can be used by non-experts. |
| Approach: | They propose a tool to detect generated text using a set of statistical methods that can be used by non-experts. |
| Outcome: | The proposed method improves detection rate of fake text from 54% to 72% without training. |
Say It Another Way: Auditing LLMs with a User-Grounded Automated Paraphrasing Framework (2026.eacl-long)
Copied to clipboard
Clea Chataigner, Rebecca Ma, Prakhar Ganesh, Yuhao Chen, Afaf Taik, Elliot Creager, Golnoosh Farnadi
| Challenge: | Existing studies have studied prompt sensitivity by altering formatting or generating paraphrases with automated techniques. |
| Approach: | They propose a framework for generating controlled paraphrases grounded in user behaviors . they leverage linguistically informed rules and enforce quality through checks on instruction adherence . |
| Outcome: | The proposed framework is able to detect weaknesses in large language models . it leverages linguistically informed rules and enforces quality through checks on instruction adherence, semantic similarity, and realism. |
Explain the Synth: Interpretable Evaluation of LLM Data Synthesis (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly used to generate tabular data. |
| Approach: | They propose a framework that uses a rule-based model as a shared explanatory language to examine the explanation of real versus synthetic data. |
| Outcome: | The proposed framework compares the explanatory structure induced by real versus synthetic data. |
Automatic Detection of Machine Generated Text: A Critical Survey (2020.coling-main)
Copied to clipboard
| Challenge: | Current text generative models excel in producing text that matches the style of human language reasonably well. |
| Approach: | They conduct an in-depth error analysis of the state-of-the-art detector and discuss research directions to guide future work in this exciting area. |
| Outcome: | The proposed detectors can distinguish between human and text generated by the model and can be used to generate fake news and fake product reviews. |
Detecting Machine-Generated Text: Techniques and Challenges (2024.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial focuses on machine-generated text and deepfakes. |
| Approach: | This tutorial aims to provide a comprehensive overview of text detection techniques . it will focus on machine-generated text and deepfakes . |
| Outcome: | This tutorial focuses on machine-generated text and deepfakes. |
Exploring Data Augmentation for Code Generation Tasks (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent advances in natural language processing have impacted how models are trained for programming language tasks. |
| Approach: | They propose to use augmentation methods that yield consistent improvements in code translation and summarization by up to 6.9% and 7.5% respectively. |
| Outcome: | The proposed methods improve translation and summarization by 6.9% and 7.5% respectively. |