| Challenge: | Large language models (LLMs) have advanced the field of NLP significantly, but deploying them for downstream applications is still challenging due to cost, responsiveness, control, or concerns around privacy and security. |
| Approach: | They propose a unified data creation pipeline that requires only a single formatting example. |
| Outcome: | The proposed pipeline can generate data with a single formatting example. |
Similar Papers
Empowering Large Language Models for Textual Data Augmentation (2024.findings-acl)
Copied to clipboard
| Challenge: | True. True. False |
| Approach: | False slants are proposed to generate a large pool of augmentation instructions and select the most suitable task-informed instructions. |
| Outcome: | False omissions: the proposed approach consistently generates augmented data with better quality compared to non-LLM and LLM-based data augmentation methods. |
The Data Frontier for Large Language Models: Selection, Synthesis, and Tools (2026.acl-tutorials)
Copied to clipboard
| Challenge: | acquiring and curating high-quality training data remains a significant bottleneck . acquiring such high-quality data is a key challenge for researchers and practitioners . |
| Approach: | This tutorial provides a comprehensive and practical guide to the state-of-the-art in data research directions for LLMs. |
| Outcome: | The tutorial covers methods for curating the most valuable information from vast, noisy datasets and the synthetic data revolution. |
A Rigorous Evaluation of LLM Data Generation Strategies for Low-Resource Languages (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used to generate synthetic textual data for training smaller specialized models. |
| Approach: | They evaluate the performance of large language models and their generation strategies in 11 different languages using 3 NLP tasks and 4 open-source LLMs. |
| Outcome: | The proposed generation strategies and their combinations yield strong results across 11 languages, including several extremely low-resource ones. |
Data and Model Centric Approaches for Expansion of Large Language Models to New languages (2025.emnlp-tutorials)
Copied to clipboard
| Challenge: | Existing LLMs mainly support English alongside a handful of high resource languages . this leaves a major gap for most low-resource languages despite increasing pace of research . |
| Approach: | This tutorial examines approaches to expand the language coverage of LLMs . they look at tokenizer training, pre-training, instruction tuning, alignment, evaluation, etc. |
| Outcome: | This tutorial examines approaches to expand the language coverage of LLMs . it provides an efficient and viable path to bring LLM technologies to low-resource languages . |
Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventions (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) can be used to generate text data for training and evaluating other models. |
| Approach: | They propose to use logit suppression and temperature sampling to diversify text generation but at the cost of data accuracy. |
| Outcome: | The proposed approach can increase diversity but at the cost of data accuracy. |
High-quality Data-to-Text Generation for Severely Under-Resourced Languages with Out-of-the-box Large Language Models (2024.findings-eacl)
Copied to clipboard
| Challenge: | Pretrained large language models (LLMs) can bridge the performance gap for under-resourced languages by substantial margins, as measured by both automatic and human evaluations. |
| Approach: | They propose to use pretrained large language models to bridge this gap by automating and evaluating data-to-text generation in under-resourced languages. |
| Outcome: | The proposed model can set the state of the art for under-resourced languages by substantial margins, as measured by both automatic and human evaluations. |
Learning with Less: Knowledge Distillation from Large Language Models via Unlabeled Data (2025.findings-naacl)
Copied to clipboard
Juanhui Li, Sreyashi Nag, Hui Liu, Xianfeng Tang, Sheikh Muhammad Sarwar, Limeng Cui, Hansu Gu, Suhang Wang, Qi He, Jiliang Tang
| Challenge: | Large Language Models (LLMs) have demonstrated superior language understanding abilities in many real-world NLP applications. |
| Approach: | They propose a learning-based sample selection method that incorporates signals from both teacher and student to enhance model performance. |
| Outcome: | The proposed method improves model performance across datasets with higher data efficiency. |
Self-training Large Language Models through Knowledge Detection (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) often require extensive labeled datasets and training compute to achieve impressive performance across downstream tasks. |
| Approach: | They propose a self-training paradigm where the LLM curates its own labels and selectively trains on unknown data samples identified through a reference-free consistency method. |
| Outcome: | The proposed model reduces the dependency on large labeled datasets and mitigates catastrophic forgetting in out-of-distribution benchmarks. |
LLMaAA: Making Large Language Models as Active Annotators (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing supervised learning methods in natural language processing require large amounts of data. |
| Approach: | They propose an active learning loop that takes LLMs as annotators and puts them into an active loop to determine what to annotate efficiently. |
| Outcome: | The proposed model outperforms existing models with few-shot performance in two NLP tasks. |
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes (2023.findings-acl)
Copied to clipboard
Cheng-Yu Hsieh, Chun-Liang Li, Chih-kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alex Ratner, Ranjay Krishna, Chen-Yu Lee, Tomas Pfister
| Challenge: | Deploying large language models (LLMs) is difficult because they are memory inefficient and compute-intensive for practical applications. |
| Approach: | They propose a mechanism that fine tunes or distills small models that outperform LLMs . they use human labels to fine tune models or LLM-generated labels to train models . |
| Outcome: | The proposed method outperforms LLMs by using fewer training examples compared to few-shot prompted models using substantially smaller model sizes. |