| Challenge: | Neural-based approaches to natural language generation are data-hungry and difficult to adopt in real-world applications. |
| Approach: | They propose a task of few-shot natural language generation from structured data or knowledge to generate coherent sentences from input data and language modeling to compose coherent sentences. |
| Outcome: | The proposed approach outperforms the strongest baseline approach by over 8.0 BLEU points improvement. |
Similar Papers
AugNLG: Few-shot Natural Language Generation using Self-trained Data Augmentation (2021.acl-long)
Copied to clipboard
| Challenge: | Large-scale conversational systems typically generate unnatural, robotic responses using template-based approaches. |
| Approach: | They propose a data augmentation approach that combines a self-trained neural retrieval model with a few-shot learned NLU model to automatically create MR-to-Text data from open-domain texts. |
| Outcome: | The proposed approach outperforms the state-of-the-art methods on the FewshotWOZ data in both BLEU and Slot Error Rate. |
Few-shot Knowledge Graph-to-Text Generation with Pretrained Language Models (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models for KG-to-text generation are based on pretrained language models. |
| Approach: | They propose to automatically generate a text that describes the facts in knowledge graph (KG) they leverage the excellent capacities of pretrained language models (PLMs) in language understanding and generation. |
| Outcome: | The proposed model outperforms all comparison methods on fully-supervised and fewshot settings. |
Few-shot Natural Language Generation for Task-Oriented Dialog (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for NLG depend on heavily annotated data, which is infeasible for new domains. |
| Approach: | They propose a system that converts a dialog act into a response in natural language . they propose 'nuclear language generation' to simulate a few-shot learning setting . |
| Outcome: | The proposed model outperforms existing methods on a large set of annotated datasets. |
Making Pre-trained Language Models Better Few-shot Learners (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that the GPT-3 model can perform few-shots on language understanding tasks with a natural-language prompt and a few task demonstrations. |
| Approach: | They propose a technique for fine-tuning language models using a few examples . they propose LM-BFF, which uses prompt-based fine-uning and a pipeline for automating prompt generation . |
| Outcome: | The proposed approach outperforms standard fine-tuning procedures on a range of NLP tasks. |
Zero- and Few-Shot NLP with Pretrained Language Models (2022.acl-tutorials)
Copied to clipboard
| Challenge: | a tutorial aims to introduce NLP researchers to the latest techniques for learning from little-to-no data . aims at bringing interested researchers up to speed about the latest and ongoing techniques . |
| Approach: | They aim to introduce techniques for learning from little-to-no data using pretrained language models. |
| Outcome: | This tutorial aims to bring interested NLP researchers up to speed about recent techniques . it will cover methods from manual engineering, better inference algorithms to better tuning methods . |
Few-shot Table-to-text Generation with Prefix-Controlled Generator (2022.coling-1)
Copied to clipboard
| Challenge: | Neural table-to-text generation approaches are data-hungry and lack labeled data. |
| Approach: | They propose a prompt-based approach for few-shot table-to-text generation using a task-specific prefix and an input-specific input prefix. |
| Outcome: | The proposed approach is able to generate table-to-text summaries with a few instances and is validated on human, book and song datasets. |
Constrained Language Models Yield Few-Shot Semantic Parsers (2021.emnlp-main)
Copied to clipboard
Richard Shin, Christopher Lin, Sam Thomson, Charles Chen, Subhro Roy, Emmanouil Antonios Platanios, Adam Pauls, Dan Klein, Jason Eisner, Benjamin Van Durme
| Challenge: | Large pretrained language models excel at generating natural language, but they are not efficient for task specific semantic parsing. |
| Approach: | They propose to use large pretrained language models as few-shot semantic parsers . they paraphrase inputs into a controlled sublanguage resembling English . |
| Outcome: | The proposed model can generate surprisingly accurate models on multiple tasks with minimal code and data. |
DSPM-NLG: A Dual Supervised Pre-trained Model for Few-shot Natural Language Generation in Task-oriented Dialogue System (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models for few-shot natural language generation are based on a dual correlation between NLG and SLU from the perspective of probability. |
| Approach: | They propose a dual supervised pre-trained model to regularize the pre-training process . they use a probabilistic approach to learn the dual correlation between NLG and SLU . |
| Outcome: | The proposed model outperforms the previous state-of-the-art models on a few-shot dataset. |
Pre-trained Language Models Can be Fully Zero-Shot Learners (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to pre-trained language models require fine-tuning on labeled datasets or manually constructing proper prompts. |
| Approach: | They propose a nonparametric prompting PLM for fully zero-shot language understanding . they compare it to previous methods for text classification and text entailment . |
| Outcome: | The proposed method outperforms previous methods on diverse tasks. |
Few-Shot Text Generation with Natural Language Instructions (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to text generation combine task descriptions and examples with supervised learning. |
| Approach: | They propose a method for text generation that is based on pattern-exploiting training. |
| Outcome: | The proposed approach improves on several summarization and headline generation datasets. |