KGPT: Knowledge-Grounded Pre-Training for Data-to-Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for data-to-text generation rely on labeled data, which is costly to acquire and limits their application to new tasks and domains. |
| Approach: | They propose to leverage pre-training and transfer learning to address this problem by leveraging a general knowledge-grounded generation model and a knowledge-based model. |
| Outcome: | The proposed model can generate knowledge-enriched text on a knowledge-grounded text corpus crawled from the web in three settings. |
Similar Papers
Few-shot Knowledge Graph-to-Text Generation with Pretrained Language Models (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models for KG-to-text generation are based on pretrained language models. |
| Approach: | They propose to automatically generate a text that describes the facts in knowledge graph (KG) they leverage the excellent capacities of pretrained language models (PLMs) in language understanding and generation. |
| Outcome: | The proposed model outperforms all comparison methods on fully-supervised and fewshot settings. |
Pretrain-KGE: Learning Knowledge Representation from Pretrained Language Models (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge graph embedding models suffer from limited knowledge representation due to sparse and noisy dataset annotations. |
| Approach: | They propose to use pretrained language models to enhance knowledge representation by leveraging world knowledge from pretrained models. |
| Outcome: | Extensive experiments show that the proposed framework can improve results over existing models. |
Enhancing Neural Data-To-Text Generation Models with External Background Knowledge (D19-1)
Copied to clipboard
| Challenge: | Recent neural models for data-to-text generation rely on parallel pairs of data and text to learn writing knowledge. |
| Approach: | They propose to enhance neural models with external knowledge to improve fidelity of generated text. |
| Outcome: | The proposed model improves on Wikipedia infobox-to-text datasets on 21 datasets. |
GAP: A Graph-aware Language Model Framework for Knowledge Graph-to-Text Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Recent improvements in KG-to-text generation are due to additional pre-training tasks . these tasks require extensive computational resources while only suggesting marginal improvements. |
| Approach: | They propose a mask structure to capture neighborhood information and a type encoder that adds a bias to the graph-attention weights depending on the connection type. |
| Outcome: | The proposed model outperforms state-of-the-art models while requiring no additional pre-training tasks. |
Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning (2023.acl-long)
Copied to clipboard
Alexander Hanbo Li, Mingyue Shang, Evangelia Spiliopoulou, Jie Ma, Patrick Ng, Zhiguo Wang, Bonan Min, William Yang Wang, Kathleen McKeown, Vittorio Castelli, Dan Roth, Bing Xiang
| Challenge: | Existing methods for data-to-text generation focus on specific types of structured data. |
| Approach: | They propose a method that provides a unified representation that can handle various forms of structured data such as tables, knowledge graph triples, and meaning representations. |
| Outcome: | The proposed method improves zero-shot and few-shot scenarios and can adapt to new structured data. |
Eliciting Knowledge from Large Pre-Trained Models for Unsupervised Knowledge-Grounded Conversation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large-scale pre-training provide large models with the potential to learn knowledge from the raw text. |
| Approach: | They propose a posterior-based reweighing and noisy training strategy to exploit generated knowledge in dialogue generation. |
| Outcome: | Empirical results show that the proposed methods outperform the state-of-the-art methods in unsupervised knowledge-grounded conversation. |
Commonsense Knowledge Transfer for Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in pre-trained language models have transformed the landscape of natural language processing. |
| Approach: | They propose a framework to transfer commonsense knowledge stored in a neural commonsensing model to a general-purpose pre-trained language model. |
| Outcome: | Empirical results show that the proposed framework improves the model’s performance on downstream tasks that require commonsense reasoning. |
Neural Data-to-Text Generation with LM-based Text Augmentation (2021.eacl-main)
Copied to clipboard
| Challenge: | Neural data-to-text generation is a difficult task for many new applications because of a lack of training data. |
| Approach: | They propose a few-shot approach that augments the data available for training by generating new text samples based on replacing specific values by alternative ones from the same category and pairing the new text with data samples. |
| Outcome: | The proposed approach outperforms fully supervised sequence-to-sequence models with less than 10% of the training set on both datasets. |
Neural Pipeline for Zero-Shot Data-to-Text Generation (2022.acl-long)
Copied to clipboard
| Challenge: | In data-to-text generation, training on in-domain data leads to overfitting and repeating training data noise. |
| Approach: | They propose to train pretrained language models on general-domain text-based operations by transforming single-item descriptions with modules trained on ordering, aggregation, and paragraph compression. |
| Outcome: | The proposed approach enables D2T generation from RDF triples in zero-shot settings. |
RecGPT: Generative Pre-training for Text-based Recommendation (2024.acl-short)
Copied to clipboard
| Challenge: | Existing models for text-based recommendation lack data sparsity and flexibility to capture fluctuations in user preferences over time. |
| Approach: | They present the first domain-adapted and fully-trained large language model for text-based recommendation. |
| Outcome: | The proposed model outperforms baseline models on rating prediction and sequential recommendation tasks. |