Few-shot Knowledge Graph-to-Text Generation with Pretrained Language Models (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models for KG-to-text generation are based on pretrained language models. |
| Approach: | They propose to automatically generate a text that describes the facts in knowledge graph (KG) they leverage the excellent capacities of pretrained language models (PLMs) in language understanding and generation. |
| Outcome: | The proposed model outperforms all comparison methods on fully-supervised and fewshot settings. |
Similar Papers
Few-Shot NLG with Pre-Trained Language Model (2020.acl-main)
Copied to clipboard
| Challenge: | Neural-based approaches to natural language generation are data-hungry and difficult to adopt in real-world applications. |
| Approach: | They propose a task of few-shot natural language generation from structured data or knowledge to generate coherent sentences from input data and language modeling to compose coherent sentences. |
| Outcome: | The proposed approach outperforms the strongest baseline approach by over 8.0 BLEU points improvement. |
Few-Shot Text Generation with Natural Language Instructions (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to text generation combine task descriptions and examples with supervised learning. |
| Approach: | They propose a method for text generation that is based on pattern-exploiting training. |
| Outcome: | The proposed approach improves on several summarization and headline generation datasets. |
GAP: A Graph-aware Language Model Framework for Knowledge Graph-to-Text Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Recent improvements in KG-to-text generation are due to additional pre-training tasks . these tasks require extensive computational resources while only suggesting marginal improvements. |
| Approach: | They propose a mask structure to capture neighborhood information and a type encoder that adds a bias to the graph-attention weights depending on the connection type. |
| Outcome: | The proposed model outperforms state-of-the-art models while requiring no additional pre-training tasks. |
Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning (2023.acl-long)
Copied to clipboard
Alexander Hanbo Li, Mingyue Shang, Evangelia Spiliopoulou, Jie Ma, Patrick Ng, Zhiguo Wang, Bonan Min, William Yang Wang, Kathleen McKeown, Vittorio Castelli, Dan Roth, Bing Xiang
| Challenge: | Existing methods for data-to-text generation focus on specific types of structured data. |
| Approach: | They propose a method that provides a unified representation that can handle various forms of structured data such as tables, knowledge graph triples, and meaning representations. |
| Outcome: | The proposed method improves zero-shot and few-shot scenarios and can adapt to new structured data. |
Zero- and Few-Shot NLP with Pretrained Language Models (2022.acl-tutorials)
Copied to clipboard
| Challenge: | a tutorial aims to introduce NLP researchers to the latest techniques for learning from little-to-no data . aims at bringing interested researchers up to speed about the latest and ongoing techniques . |
| Approach: | They aim to introduce techniques for learning from little-to-no data using pretrained language models. |
| Outcome: | This tutorial aims to bring interested NLP researchers up to speed about recent techniques . it will cover methods from manual engineering, better inference algorithms to better tuning methods . |
Syntax Controlled Knowledge Graph-to-Text Generation with Order and Semantic Consistency (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing knowledge graph-to-text generation methods focus on sequence-to sequence generation, but the linearized order of KG is obtained through a heuristic search without data-driven optimization. |
| Approach: | They propose to generate easy-to-understand sentences from the knowledge graph . they incorporate part-of-speech syntactic tags to constrain the positions to copy words from the KG and employ a semantic context scoring function to evaluate the semantic fitness for each word in its local context. |
| Outcome: | The proposed method achieves state-of-the-art on two datasets, WebNLG and DART, and achieves high consistency. |
Pre-trained Language Models Can be Fully Zero-Shot Learners (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to pre-trained language models require fine-tuning on labeled datasets or manually constructing proper prompts. |
| Approach: | They propose a nonparametric prompting PLM for fully zero-shot language understanding . they compare it to previous methods for text classification and text entailment . |
| Outcome: | The proposed method outperforms previous methods on diverse tasks. |
Knowledge Graph Generation From Text (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for generating text from text are limited due to non-unique graph representation, complex node structure, large output spaces and limited parallel training data. |
| Approach: | They propose a novel end-to-end multi-stage Knowledge Graph generation system from textual inputs that separates the overall process into two stages. |
| Outcome: | The proposed system outperforms existing methods on a WebNLG 2020 Challenge dataset and on TekGen datasets. |
Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to few-shot text classification require domain expertise and an understanding of the language model's abilities to define the mapping between words and labels. |
| Approach: | They propose a method that converts textual inputs to cloze questions that contain some form of task description and processes them with a pretrained language model to map the predicted words to labels. |
| Outcome: | The proposed approach performs almost as well as hand-crafted label-to-word mappings for a number of tasks with small amounts of training data. |
Constrained Language Models Yield Few-Shot Semantic Parsers (2021.emnlp-main)
Copied to clipboard
Richard Shin, Christopher Lin, Sam Thomson, Charles Chen, Subhro Roy, Emmanouil Antonios Platanios, Adam Pauls, Dan Klein, Jason Eisner, Benjamin Van Durme
| Challenge: | Large pretrained language models excel at generating natural language, but they are not efficient for task specific semantic parsing. |
| Approach: | They propose to use large pretrained language models as few-shot semantic parsers . they paraphrase inputs into a controlled sublanguage resembling English . |
| Outcome: | The proposed model can generate surprisingly accurate models on multiple tasks with minimal code and data. |