Explain the Synth: Interpretable Evaluation of LLM Data Synthesis (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly used to generate tabular data. |
| Approach: | They propose a framework that uses a rule-based model as a shared explanatory language to examine the explanation of real versus synthetic data. |
| Outcome: | The proposed framework compares the explanatory structure induced by real versus synthetic data. |
Similar Papers
On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) provide a data-centric solution to alleviate limitations of real-world data with synthetic data generation. |
| Approach: | They propose a generic workflow for LLM-driven synthetic data generation. |
| Outcome: | The proposed workflows highlight gaps in existing research and outline avenues for future studies. |
An LLM-Based Approach for Insight Generation in Data Analysis (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to generate insightful data from databases are time-consuming and resource-intensive. |
| Approach: | They propose a method that leverages Large Language Models to automatically generate textual insights from databases. |
| Outcome: | The proposed approach generates more insightful insights than other approaches while maintaining correctness. |
Synthetic Data for Evaluation: Supporting LLM-as-a-Judge Workflows with EvalAssist (2025.emnlp-demos)
Copied to clipboard
Martín Santillán Cooper, Zahra Ashktorab, Hyo Jin Do, Erik Miehling, Werner Geyer, Jasmina Gajcin, Elizabeth M. Daly, Qian Pan, Michael Desmond
| Challenge: | EvalAssist is a web-based application designed to assist human-centered evaluation of language model outputs. |
| Approach: | They propose a synthetic data generation tool integrated into EvalAssist to assist human-centered evaluation of language model outputs. |
| Outcome: | The proposed tool supports flexible prompting, RAG-based grounding, persona diversity, and iterative generation workflows. |
Bridging Internal Consistency and External Alignment: A Causal and Dynamic Interpretability Framework for LLM Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing interpretability methods focus on internal and external aspects of the model . existing explanations often focus on surface correlations or static dependencies . |
| Approach: | They propose a causal and dynamic interpretability framework for Large Language Models . they characterize backdoor-adjusted causal effects of generated prefix and prompt . |
| Outcome: | The proposed framework provides a unified causal view of internal consistency and external alignment in LLM generation dynamics. |
SynthTextEval: Synthetic Text Data Generation and Evaluation for High-Stakes Domains (2025.emnlp-demos)
Copied to clipboard
Krithika Ramesh, Daniel Smolyak, Zihao Zhao, Nupoor Gandhi, Ritu Agarwal, Margrét V. Bjarnadóttir, Anjalie Field
| Challenge: | SynthTextEval is a toolkit for conducting comprehensive evaluations of synthetic text. |
| Approach: | They propose a toolkit for conducting comprehensive evaluations of synthetic text using large language models. |
| Outcome: | The proposed toolkit can be run over any dataset, but it is aimed at two high-stakes domains: healthcare and law. |
Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extraction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have great potential for synthetic data generation. |
| Approach: | They show that large language models can generate useful data even for complex tasks . they use a symmetric task difficulty asymmetry to prompt an LLM to generate plausible input text for a target output structure. |
| Outcome: | The proposed approach outperforms existing models by a substantial margin on closed information extraction tasks with 1.8M data points and 770M parameters. |
ExPerT: Effective and Explainable Evaluation of Personalized Long-Form Text Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Evaluating personalized text generated by large language models is challenging, as only the LLM user, i.e. prompt author, can reliably assess the output. |
| Approach: | They propose an explainable reference-based evaluation framework that leverages an LLM to extract atomic aspects and their evidences from the generated and reference texts, match the aspects, and evaluate their alignment based on content and writing style. |
| Outcome: | The proposed framework achieves a 7.2% improvement in alignment with human judgments compared to the state-of-the-art evaluation methods. |
SciText2Eq: Assessing LLMs for Explainable Equation Generation for Scientific Creativity (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior work has addressed problems in unstructured grounding, multi-equation dependency, and human-aligned evaluation. |
| Approach: | They construct a dataset of scientific texts and evaluate it using an explainable equation generation workflow using automatic metrics and human judgments. |
| Outcome: | The proposed model achieves moderate performance on lexical and syntactic similarity, but struggles with semantic accuracy. |
When Backdoors Speak: Understanding LLM Backdoor Attacks Through Model-Generated Explanations (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that Large Language Models (LLMs) are susceptible to backdoor attacks, where triggers embedded in poisoned data can maliciously alter LLMs’ behaviors. |
| Approach: | They propose to leverage LLMs' generative capabilities to generate human-readable explanations for their decisions, enabling direct comparisons between explanations of clean and poisoned data. |
| Outcome: | The proposed model produces coherent explanations for clean inputs but logically flawed explanations on poisoned data. |
Evaluating Language Models as Synthetic Data Generators (2025.acl-long)
Copied to clipboard
Seungone Kim, Juyoung Suk, Xiang Yue, Vijay Viswanathan, Seongyun Lee, Yizhong Wang, Kiril Gashteovski, Carolin Lawrence, Sean Welleck, Graham Neubig
| Challenge: | Prior studies have focused on developing effective data generation methods, but lack systematic comparison of different LMs as data generators in a unified setting. |
| Approach: | They propose to use a benchmark to compare language models' data generation abilities against a set of standardized settings and metrics. |
| Outcome: | The proposed benchmark provides standardized settings and metrics to evaluate LMs’ data generation abilities. |