An Enhanced Knowledge Injection Model for Commonsense Generation (2020.coling-main)
Copied to clipboard
Zhihao Fan, Yeyun Gong, Zhongyu Wei, Siyuan Wang, Yameng Huang, Jian Jiao, Xuanjing Huang, Nan Duan, Ruofei Zhang
| Challenge: | a recent study shows that digging the relationship of concepts from scratch is non-trivial for commonsense generation tasks. |
| Approach: | They use a retrieve-and-edit framework to retrieve a prototype with these concepts . they use qt and qq to generate commonsense questions at scale . |
| Outcome: | The proposed method significantly improves the performance on commonsense generation tasks. |
Similar Papers
A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation (2020.tacl-1)
Copied to clipboard
| Challenge: | Existing models for story generation suffer from repetition, logic conflicts, and lack of long-range coherence . |
| Approach: | They propose to utilize commonsense knowledge from external knowledge bases to generate reasonable stories by multi-task learning. |
| Outcome: | The proposed model can generate more reasonable stories than state-of-the-art models, compared with existing models, showing that it can capture useful semantic and syntactic features. |
Retrieval Enhanced Model for Commonsense Generation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing frameworks for commonsense generation are lacking for pre-trained models. |
| Approach: | They propose a framework that uses concept matching to retrieve prototype sentences and trainable sentence retriever to enhance pre-training and fine-tuning. |
| Outcome: | The proposed framework achieves state-of-the-art on the large-scale Common-Gen benchmark. |
Generated Knowledge Prompting for Commonsense Reasoning (2022.acl-long)
Copied to clipboard
Jiacheng Liu, Alisa Liu, Ximing Lu, Sean Welleck, Peter West, Ronan Le Bras, Yejin Choi, Hannaneh Hajishirzi
| Challenge: | Existing methods for commonsense reasoning rely on high-quality knowledge, but they are often dominated by large-scale pretrained models that are fine-tuned on a target benchmark. |
| Approach: | They develop generated knowledge prompting which generates knowledge from a language model and provides it as additional input when answering a question. |
| Outcome: | The proposed method improves state-of-the-art models on four commonsense reasoning tasks. |
Data-Efficient Concept Extraction from Pre-trained Language Models for Commonsense Explanation Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to extract concepts from pre-trained language models are not suitable for commonsense explanation generation. |
| Approach: | They propose a method to extract the key explanation concept from pre-trained language models by fine-tuning it with 20% training data and using a metric to evaluate the retrieved concepts. |
| Outcome: | The proposed method improves evaluation metrics over pre-trained language models and the existing models. |
Commonsense Knowledge Transfer for Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in pre-trained language models have transformed the landscape of natural language processing. |
| Approach: | They propose a framework to transfer commonsense knowledge stored in a neural commonsensing model to a general-purpose pre-trained language model. |
| Outcome: | Empirical results show that the proposed framework improves the model’s performance on downstream tasks that require commonsense reasoning. |
CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained language models perform well on commonsense-reasoning benchmark datasets, but building machines with commonsence to compose plausible sentences remains challenging. |
| Approach: | They propose a constrained text generation task for generative commonsense reasoning that generates a coherent sentence using common concepts. |
| Outcome: | The proposed task generates a coherent sentence describing an everyday scenario using common concepts over 35k concept-sets. |
Bridging the Gap between Pre-Training and Fine-Tuning for Commonsense Generation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods focusing on this task usually concatenate the concatened concepts words as the inputs of a pre-trained language model (PLM) however, in pre-training, the input is often corrupted sentences with correct word order. |
| Approach: | They propose a two-stage framework to improve the ability of pre-trained language models to deal with masked sentences with incorrect word order and a special token to make the input distribution more similar to the one used in pre-training. |
| Outcome: | The proposed method is able to generate a sentence containing all given concepts and correctly describe the relations between concepts. |
Generating Commonsense Explanation by Extracting Bridge Concepts from Reasoning Paths (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing tasks that use commonsense reasoning as multi-choice reading comprehension lack direct assessment to machine commonsence and impede its practicability to realistic scenarios. |
| Approach: | They propose a method that first extracts the underlying concepts which are served as bridges in the reasoning chain and then integrates these concepts to generate the final explanation. |
| Outcome: | The proposed model outperforms the state-of-the-art models in automatic and human evaluation. |
Synthetic Knowledge Ingestion: Towards Knowledge Refinement and Injection for Enhancing Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models capture factual knowledge across a wide range of domains, but refining their capabilities on previously seen knowledge remains a challenge. |
| Approach: | They propose a synthetic knowledge ingestion method that leverages fine-grained synthesis and interleaved generation to construct high-quality data representations from raw knowledge sources. |
| Outcome: | The proposed method outperforms baseline methods on question-answering tasks spanning finance, biomedicine, and open-generation domains. |
COMET: Commonsense Transformers for Automatic Knowledge Graph Construction (P19-1)
Copied to clipboard
| Challenge: | Existing studies on commonsense knowledge base construction only store loosely structured open-text descriptions of knowledge. |
| Approach: | They propose a commonsense knowledge base construction model that generates rich commonsensense descriptions in natural language. |
| Outcome: | The proposed models can generate rich and diverse commonsense descriptions in natural language. |