| Challenge: | Existing frameworks for commonsense generation are lacking for pre-trained models. |
| Approach: | They propose a framework that uses concept matching to retrieve prototype sentences and trainable sentence retriever to enhance pre-training and fine-tuning. |
| Outcome: | The proposed framework achieves state-of-the-art on the large-scale Common-Gen benchmark. |
Similar Papers
CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained language models perform well on commonsense-reasoning benchmark datasets, but building machines with commonsence to compose plausible sentences remains challenging. |
| Approach: | They propose a constrained text generation task for generative commonsense reasoning that generates a coherent sentence using common concepts. |
| Outcome: | The proposed task generates a coherent sentence describing an everyday scenario using common concepts over 35k concept-sets. |
MORE: Multi-mOdal REtrieval Augmented Generative Commonsense Reasoning (2024.findings-acl)
Copied to clipboard
| Challenge: | Language Models (LLMs) have gained increasing prominence in artificial intelligence, especially Large Language Model (LLm) due to the well-recognized reporting bias, the recording of commonsense information is significantly less than its existence in reality. |
| Approach: | They propose a Multi-mOdal REtrieval framework to leverage both text and images to enhance commonsense ability of language models. |
| Outcome: | The proposed framework can leverage both text and images to enhance commonsense ability of language models. |
An Enhanced Knowledge Injection Model for Commonsense Generation (2020.coling-main)
Copied to clipboard
Zhihao Fan, Yeyun Gong, Zhongyu Wei, Siyuan Wang, Yameng Huang, Jian Jiao, Xuanjing Huang, Nan Duan, Ruofei Zhang
| Challenge: | a recent study shows that digging the relationship of concepts from scratch is non-trivial for commonsense generation tasks. |
| Approach: | They use a retrieve-and-edit framework to retrieve a prototype with these concepts . they use qt and qq to generate commonsense questions at scale . |
| Outcome: | The proposed method significantly improves the performance on commonsense generation tasks. |
Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work has proposed to improve relevance modeling by having large language models actively involved in retrieval, i.e., to guide retrieval with generation. |
| Approach: | They propose to have large language models actively involved in retrieval to guide retrieval with generation. |
| Outcome: | The proposed method synergizes retrieval and generation in an iterative manner, and can generate better results in subsequent iterations. |
A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation (2020.tacl-1)
Copied to clipboard
| Challenge: | Existing models for story generation suffer from repetition, logic conflicts, and lack of long-range coherence . |
| Approach: | They propose to utilize commonsense knowledge from external knowledge bases to generate reasonable stories by multi-task learning. |
| Outcome: | The proposed model can generate more reasonable stories than state-of-the-art models, compared with existing models, showing that it can capture useful semantic and syntactic features. |
Bridging the Gap between Pre-Training and Fine-Tuning for Commonsense Generation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods focusing on this task usually concatenate the concatened concepts words as the inputs of a pre-trained language model (PLM) however, in pre-training, the input is often corrupted sentences with correct word order. |
| Approach: | They propose a two-stage framework to improve the ability of pre-trained language models to deal with masked sentences with incorrect word order and a special token to make the input distribution more similar to the one used in pre-training. |
| Outcome: | The proposed method is able to generate a sentence containing all given concepts and correctly describe the relations between concepts. |
Revisiting Generative Commonsense Reasoning: A Pre-Ordering Approach (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to generative commonsense reasoning hypothesize that pre-trained models lack sufficient parametric knowledge for this task. |
| Approach: | They propose to use order-agnostic input to elaborately manipulate the order of the given concepts before generation to evaluate their commonsense knowledge. |
| Outcome: | The proposed approach outperforms more sophisticated models with a lot of external data and resources in the task of generating a logical sentence from a set of concepts. |
Retrieval Augmentation for Commonsense Reasoning: A Unified Approach (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for retrieving encyclopedic knowledge lack a large corpus and effective commonsense retriever. |
| Approach: | They propose a framework for retrieval-augmented commonsense reasoning with a large commonsensense corpus and a commonseense retriever. |
| Outcome: | The proposed framework outperforms existing methods on commonsense reasoning tasks. |
ParetoRAG: Leveraging Sentence-Context Attention for Robust and Efficient Retrieval-Augmented Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Retrieval-augmented generation systems face persistent challenges in retrieval inefficiency and the inability of LLMs to filter out irrelevant information. |
| Approach: | They propose an unsupervised framework that optimizes RAG systems through sentence-level refinement guided by the Pareto principle. |
| Outcome: | The proposed framework achieves dual improvements in retrieval precision and generation quality without additional training or API resources while using only 40% of the tokens compared to traditional approaches. |
Harnessing Black-Box Control to Boost Commonsense in LM’s Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent years have seen remarkable progress in massively Pre-Trained Language Models such as GPT-3 . however, their generated outputs lack commonsense at times . |
| Approach: | They propose a framework that steers a frozen Pre-Trained Language Model towards more commonsense generation by training an auxiliary model. |
| Outcome: | The proposed framework produces plausible outputs that incorporate concepts in a meaningful way. |