FEAT: A Preference Feedback Dataset through a Cost-Effective Auto-Generation and Labeling Framework for English AI Tutoring (2025.acl-short)
Copied to clipboard
| Challenge: | Existing algorithms for teacher feedback generation are time-consuming and costly to generate manually. |
| Approach: | They propose a framework for generating teacher feedback using LLMs and humans . they construct three datasets that are time-consuming and costly to generate manually . results show that incorporating a small portion of DM leads to superior performance . |
| Outcome: | The proposed framework performs better on three datasets compared to human-generated feedback and LLM-generated datasets. |
Similar Papers
C2: Scalable Auto-Feedback for LLM-based Chart Generation (2025.naacl-long)
Copied to clipboard
Woosung Koh, Janghan Yoon, MinHyung Lee, Youngjin Song, Jaegwan Cho, Jaehyun Kang, Taehyeon Kim, Se-Young Yun, Youngjae Yu, Bongshin Lee
| Challenge: | generating high-quality charts with Large Language Models presents significant challenges due to limited data and the high cost of curation. |
| Approach: | They propose a referencefree automatic feedback generator to generate high-quality charts with Large Language Models. |
| Outcome: | The proposed framework outperforms baselines and shows that it significantly improves data diversity. |
Beyond Sample-Level Feedback: Using Reference-Level Feedback to Guide Data Synthesis (2026.eacl-long)
Copied to clipboard
| Challenge: | High-quality instruction-tuning data is crucial for Large Language Models (LLMs) but it imposes a quality ceiling where models trained on the data cannot outperform the LLM generating it. |
| Approach: | They propose a paradigm that extracts desirable characteristics from carefully curated reference samples to guide the synthesis of higher-quality instruction-response pairs. |
| Outcome: | The proposed paradigm outperforms traditional sample-level feedback methods and generalizes across model architectures. |
UICoder: Finetuning Large Language Models to Generate User Interface Code through Automated Feedback (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to improve UI code generation rely on expensive human feedback or distilling a proprietary model. |
| Approach: | They propose to use automated feedback to guide large language models to generate UI code . they use a large synthetic dataset to generate improved models and refine them . |
| Outcome: | The proposed model outperforms baseline models and larger proprietary models . the model outpersforms models with automated metrics and human preferences . |
Learning to Verify Summary Facts with Fine-Grained LLM Feedback (2025.coling-main)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have significantly enhanced the text summarization performance, but hallucination issues still occur in summaries. |
| Approach: | They propose a large-scale dataset containing fine-grained factual feedback on summaries that can be fine tuned by using Large Language Models (LLMs) they employ 10 distinct LLMs for diverse summary generation and Llama-3-70B-Instruct for feedback. |
| Outcome: | The proposed model outperforms models trained on smaller human-annotated datasets while maintaining high performance. |
Closing the Loop: Learning to Generate Writing Feedback via Language Model Simulated Student Revisions (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in language models (LMs) have made it possible to automatically generate feedback that is actionable and well-aligned with human-specified attributes. |
| Approach: | They propose a tool that PROduces Feedback via learning from LM simulated student revisions and propose to iteratively optimize the feedback generator by directly maximizing the effectiveness of students’ overall revising performance. |
| Outcome: | The proposed approach surpasses baseline methods in effectiveness of improving students’ writing and demonstrates enhanced pedagogical values, even though it was not explicitly trained for this aspect. |
Prompt Candidates, then Distill: A Teacher-Student Framework for LLM-driven Data Annotation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for data annotation use an aggressive approach prompting LLMs to determine a single gold label for each unlabeled sample. |
| Approach: | They propose a teacher-student framework that distills candidate annotations with a Small Language Model (SLM) they propose to use LLMs to generate and distill candidate annotation with slms to ensure unique labels are provided for downstream tasks. |
| Outcome: | The proposed method outperforms existing methods due to uncertainty in LLMs and is noisetolerant. |
From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning (2024.naacl-long)
Copied to clipboard
Ming Li, Yong Zhang, Zhitao Li, Jiuhai Chen, Lichang Chen, Ning Cheng, Jianzong Wang, Tianyi Zhou, Jing Xiao
| Challenge: | Large Language Models (LLMs) have revolutionized the landscape of artificial intelligence. |
| Approach: | They propose a self-guided method to identify and select cherry samples from open-source datasets, minimizing manual curation and potential cost for instruction tuning an LLM. |
| Outcome: | The proposed method enables LLMs to identify discrepancies between expected responses and intrinsic generation capability, and a marked uptick in model training efficiency. |
Rapidly Developing High-quality Instruction Data and Evaluation Benchmark for Large Language Models with Minimal Human Effort: A Case Study on Japanese (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have aimed to refine their capacity to accurately follow human instructions and navigate intricate scenarios. |
| Approach: | They propose a method that uses a set of instructions to translate English into Japanese and then generates Japanese instruction data using GPT-4. |
| Outcome: | The proposed method outperforms Japanese-Alpaca models in the evaluation benchmarks without human references. |
DS2-Instruct: Domain-Specific Data Synthesis for Large Language Models Instruction Tuning (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing data synthesis methods focus on general-purpose tasks and fail to capture domain-specific terminology and reasoning patterns. |
| Approach: | They propose a framework that generates domain-specific instruction datasets without human supervision by pairing task-informed keywords with different cognitive levels from Bloom’s Taxonomy. |
| Outcome: | The proposed framework generates domain-specific instruction datasets without human supervision and achieves significant improvements over existing methods. |
Text2Chart31: Instruction Tuning for Chart Generation with Automatic Feedback (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets do not cover full range of chart types, such as 3D, volumetric, and gridded charts. |
| Approach: | They propose a hierarchical pipeline and a new dataset for chart generation that leverages the relationships within rich datasets. |
| Outcome: | The proposed method outperforms open-source models and is comparable to state-of-the-art proprietary models in data visualization tasks. |