ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that large pretrained language models can generate training data with no task-specific or cross-task data. |
| Approach: | They propose a retrieval-enhanced framework to create training data from a general-domain unlabeled corpus. |
| Outcome: | The proposed framework achieves 4.3% gain over baselines and saves 70% of time compared with baselines using large language models. |
Similar Papers
On the Zero-Shot Generalization of Machine-Generated Text Detectors (2023.findings-emnlp)
Copied to clipboard
| Challenge: | rampant proliferation of large language models generates text indistinguishable from human-written language. |
| Approach: | They train neural detectors on outputs of a new generator and test their performance on held-out generators. |
| Outcome: | The proposed detectors can be built on training data from medium-sized models. |
ZeroGen: Efficient Zero-shot Learning via Dataset Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to generate training data with pre-trained language models have been found effective in various scenarios. |
| Approach: | They propose an unsupervised zero-shot learning method that generates a dataset from scratch and trains a tiny task model under supervision of the synthesized dataset. |
| Outcome: | The proposed method is annotated-free and efficient, but can provide useful insights from the perspective of data-free model-agnostic knowledge distillation and unreferenced text generation evaluation. |
Zero-Shot Text Classification with Self-Training (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large pretrained language models have increased attention to zero-shot text classification. |
| Approach: | They propose a plug-and-play method to bridge this gap by requiring only class names along with an unlabeled dataset. |
| Outcome: | The proposed model can be trained on a natural language inference dataset and performs on dozens of unseen tasks without the need for domain expertise or trial and error. |
Zero- and Few-Shot NLP with Pretrained Language Models (2022.acl-tutorials)
Copied to clipboard
| Challenge: | a tutorial aims to introduce NLP researchers to the latest techniques for learning from little-to-no data . aims at bringing interested researchers up to speed about the latest and ongoing techniques . |
| Approach: | They aim to introduce techniques for learning from little-to-no data using pretrained language models. |
| Outcome: | This tutorial aims to bring interested NLP researchers up to speed about recent techniques . it will cover methods from manual engineering, better inference algorithms to better tuning methods . |
ZeroDL: Zero-shot Distribution Learning for Text Clustering via Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown impressive performance on downstream tasks, but if they cannot be fully described in prompts, they could fail to perform the task. |
| Approach: | They propose a method to contextualize a task toward a large language model (LLM) they use open-ended zero-shot inference from the entire dataset to aggregate the inference results and incorporate the aggregated meta-information for the actual task. |
| Outcome: | The proposed method improves text clustering tasks and improves on several datasets. |
Generation-driven Contrastive Self-training for Zero-shot Text Classification with Instruction-following LLM (2024.eacl-long)
Copied to clipboard
| Challenge: | a novel method to train a smaller model with LLMs for zero-shot text classification requires immense computational resources due to their substantial model size. |
| Approach: | They propose a method which leverages the generative power of large language models to train a smaller model. |
| Outcome: | The proposed method outperforms state-of-the-art methods when limited data is available. |
Generation-Augmented Retrieval: Rethinking the Role of Large Language Models in Zero-Shot Relation Extraction (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in Relation Extraction (RE) emphasize Zero-Shot methodologies, aiming to recognize unseen relations between entities with no annotated data. |
| Approach: | They propose a plug-in retrieval adjuster that allows rapid fine-tuning without accessing LLMs’ parameters. |
| Outcome: | The proposed model demonstrates comparable performance on multiple benchmarks. |
Label Agnostic Pre-training for Zero-shot Text Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to text classification assume a fixed set of labels . however, in real-world applications, there exists an infinite label space for describing a given text . |
| Approach: | They propose two new methods that inject aspect-level understanding into pre-trained models at train time to improve zero-shot generalization. |
| Outcome: | The proposed methods improve zero-shot generalization on a set of challenging datasets. |
A Universal Discriminator for Zero-Shot Generalization (2023.acl-long)
Copied to clipboard
| Challenge: | Generative modeling has been the dominant approach for large-scale pretraining and zeroshot generalization. |
| Approach: | They propose a discriminator that predicts whether a text sample comes from the true data distribution and which option has the highest probability of coming from the real data distribution. |
| Outcome: | The proposed discriminative approach outperforms GANs on a number of NLP tasks by 16.0%, 7.8%, and 11.5% respectively. |
Grasping the Essentials: Tailoring Large Language Models for Zero-Shot Relation Extraction (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing Relation extraction models require extensive annotated training data, which is costly and labor-intensive to collect. |
| Approach: | They propose a new zero-shot RE task where only relation definitions are provided instead of seen-unseen relation instances. |
| Outcome: | The proposed task significantly improves cost-effective zero-shot performance by large margins. |