Towards Generalizable Neuro-Symbolic Systems for Commonsense Question Answering (D19-60)
Copied to clipboard
| Challenge: | Recent approaches on non-extractive commonsense QA show increased performance . attention-based injection seems to be preferable for knowledge integration . |
| Approach: | They propose to use attention-based injection to integrate knowledge into commonsense QA models. |
| Outcome: | The proposed methods show that attention-based injection is preferable for knowledge integration, and that the degree of domain overlap plays a crucial role in determining model success. |
Similar Papers
A guide to the dataset explosion in QA, NLI, and commonsense reasoning (2020.coling-tutorials)
Copied to clipboard
| Challenge: | a tutorial aims to provide an up-to-date guide to the recent datasets . the target audience is the NLP practitioners who are lost in dozens of the recent data sets. |
| Approach: | This tutorial provides an up-to-date guide to the recent datasets . it surveys old and new methodological issues with dataset construction . |
| Outcome: | This tutorial aims to provide an up-to-date guide to the recent datasets . it surveys the old and new methodological issues with dataset construction . |
Elaboration-Generating Commonsense Question Answering at Scale (2023.acl-long)
Copied to clipboard
| Challenge: | elaborations are generated using language models that generate background knowledge that helps improve performance . human evaluations show that the quality of the generated ellaborations is high . |
| Approach: | They propose to finetune smaller language models to generate useful intermediate context . they compare a language model with an answer predictor and generate elaborations . human evaluations show that the quality of the generated ellaborations is high . |
| Outcome: | The proposed framework outperforms other models on commonsense questions on four commons sense benchmarks. |
Improving Question Answering with External Knowledge (D19-58)
Copied to clipboard
| Challenge: | ARC-Easy, ARC Challenge, and OpenBookQA use Wikipedia to augment training data . performance degrades when additional instances exhibit higher difficulty than original training data. |
| Approach: | They propose two methods for exploiting external knowledge for QA in science . they enrich the original corpus with relevant text snippets from an open-domain resource . the second method simply increases the amount of training data by appending additional in-domain instances. |
| Outcome: | The proposed methods achieve gains in accuracy of 8.1%, 13.0%, and 12.8% on science QA tasks. |
Exploring Strategies for Generalizable Commonsense Reasoning with Pre-trained Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work proposes lightweight updates to improve commonsense reasoning models . fine-tuning can cause models to overfit to task-specific data and forget knowledge gained during training . |
| Approach: | They propose to use lightweight models to update pre-trained language models to learn commonsense background knowledge. |
| Outcome: | The proposed models learn from commonsense reasoning datasets, but they are overfitted and limited generalized. |
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge (N19-1)
Copied to clipboard
| Challenge: | Recent work on question answering relies on factoid questions with little general knowledge. |
| Approach: | They propose a dataset to capture commonsense question answering with prior knowledge . they extract multiple-choice questions that discriminate between the source and target concepts . |
| Outcome: | The proposed dataset captures commonsense reasoning beyond associations . it obtains 56% accuracy, well below human performance, which is 89% . |
Social IQa: Commonsense Reasoning about Social Interactions (D19-1)
Copied to clipboard
| Challenge: | Social IQa contains 38,000 multiple choice questions for probing emotional and social intelligence in a variety of everyday situations. |
| Approach: | They propose a crowdsourcing framework that collects commonsense questions along with correct and incorrect answers about social interactions. |
| Outcome: | The proposed framework mitigates stylistic artifacts in incorrect answers by asking workers to provide the right answer to a different but related question. |
Commonsense for Generative Multi-Hop Question Answering Tasks (D18-1)
Copied to clipboard
| Challenge: | Reading comprehension QA tasks have seen a recent surge in popularity, yet most work has focused on fact-finding extractive QA. |
| Approach: | They propose a multi-hop generative task that uses a pointer-generator decoder to synthesize disjoint pieces of information within the context to generate an answer. |
| Outcome: | The proposed model performs better than previous generative models and is competitive with current state-of-the-art span prediction models. |
Improving Unsupervised Commonsense Reasoning Using Knowledge-Enabled Natural Language Inference (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent methods based on pre-trained language models have shown strong supervised performance on commonsense reasoning. |
| Approach: | They propose to use a common framework to solve commonsense reasoning tasks using a dataset from NLI. |
| Outcome: | The proposed method achieves state-of-the-art unsupervised performance on two commonsense reasoning tasks. |
Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answering (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing QA systems do not have commonsense knowledge or cannot reason with it. |
| Approach: | They propose to augment a general commonsense QA framework with a knowledgeable path generator by extrapolating existing paths from a KG with 'state-of-the-art' language model. |
| Outcome: | The generated paths are interpretable, novel, and relevant to the task. |
ZEBRA: Zero-Shot Example-Based Retrieval Augmentation for Commonsense Question Answering (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current Large Language Models (LLMs) have shown strong reasoning capabilities in commonsense question answering benchmarks, but the process underlying their success remains largely opaque. |
| Approach: | They propose a zero-shot question answering framework that combines retrieval, case-based reasoning and introspection to improve the model's performance and interpretability. |
| Outcome: | The proposed framework outperforms existing LLMs and previous knowledge integration approaches in commonsense reasoning benchmarks and achieves an average accuracy improvement of 4.5 points. |