| Challenge: | a new system that uses common sense to solve a common sense problem is developed . a winograd schema challenge and a choice of plausible alternatives are popular tests . |
| Approach: | They propose an automatic system that achieves state-of-the-art results on the Winograd Schema Challenge . they use a knowledge hunting module to gather web text for problem resolutions . |
| Outcome: | The proposed system achieves state-of-the-art on the Winograd Schema Challenge . it improves F1 performance on the full WSC by 0.21 over the previous best . |
Similar Papers
A Generalized Knowledge Hunting Framework for the Winograd Schema Challenge (N18-4)
Copied to clipboard
| Challenge: | a new system that performs well on common-sense reasoning tasks is developed . the Winograd Schema Challenge (WSC) is a popular alternative to the Turing test . |
| Approach: | They propose an automatic system that performs well on two common-sense reasoning tasks. |
| Outcome: | The proposed system improves performance on the Winograd Schema Challenge and COPA by 0.16 over the previous best. |
Combining Knowledge Hunting and Neural Language Models to Solve the Winograd Schema Challenge (P19-1)
Copied to clipboard
| Challenge: | Existing methods to solve Winograd Schema Challenge use only knowledge embedded in text . this limits the performance of such models on the WSC problems. |
| Approach: | They propose to augment existing language models with a commonsense knowledge hunting module and an explicit reasoning module to extract the needed knowledge from text. |
| Outcome: | The proposed system improves on the language model based methods by 5.53% and 7.7% on the dataset. |
How Reasonable are Common-Sense Reasoning Tasks: A Case-Study on the Winograd Schema Challenge and SWAG (D19-1)
Copied to clipboard
| Challenge: | a recent study has improved the state-of-the-art on common-sense reasoning benchmarks . a san francisco-based approach to common-ense reasoning is challenging . |
| Approach: | They propose to use common-sense reasoning benchmarks to test machine learning's common-sentence inference task SWAG to test common-mind systems. |
| Outcome: | a new study shows that improved performance on common-sense reasoning benchmarks is genuine . the proposed task is more difficult than the current one, but it is more efficient than the previous one. |
WinoWhy: A Deep Diagnosis of Essential Commonsense Knowledge for Answering Winograd Schema Challenge (2020.acl-main)
Copied to clipboard
| Challenge: | a new task is developed to categorize essential commonsense knowledge for the Winograd Schema Challenge . a previous work focused on the classification of commonsensical knowledge by pre-trained models . |
| Approach: | They propose a task that categorizes essential commonsense knowledge for the Winograd Schema Challenge . they leverage the collected reasons to identify plausible reasons from wrong ones . |
| Outcome: | The proposed task aims to shed light on commonsense knowledge needed for better reasoning . it is based on the collected reasons and a new task called WinoWhy . |
WinoLogic: A Zero-Shot Logic-based Diagnostic Dataset for Winograd Schema Challenge (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent success of neural language models on the Winograd Schema Challenge has called for further investigation of commonsense reasoning ability of these models. |
| Approach: | They propose a logic-based framework that focuses on high-quality commonsense knowledge. |
| Outcome: | The proposed framework focuses on high-quality commonsense knowledge. |
Improving Unsupervised Commonsense Reasoning Using Knowledge-Enabled Natural Language Inference (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent methods based on pre-trained language models have shown strong supervised performance on commonsense reasoning. |
| Approach: | They propose to use a common framework to solve commonsense reasoning tasks using a dataset from NLI. |
| Outcome: | The proposed method achieves state-of-the-art unsupervised performance on two commonsense reasoning tasks. |
RJE: A Retrieval-Judgment-Exploration Framework for Efficient Knowledge Graph Question Answering with LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge graph question answering (KGQA) aims to answer natural language questions using knowledge graphs. |
| Approach: | They propose a framework that retrieves refined reasoning paths and evaluates their sufficiency. |
| Outcome: | The proposed framework outperforms existing baselines while enabling small open-source LLMs to achieve competitive results without fine-tuning LLM. |
DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text (2024.findings-naacl)
Copied to clipboard
| Challenge: | Retrievalaugmented LLMs have been used to ground LLM in external knowledge . a gap exists in the current landscape regarding the effectiveness of grounding LLM on heterogeneous knowledge sources. |
| Approach: | They propose a model that uses symbolic language to generate symbolic queries . they use a dataset that is generated using predefined reasoning chains and human annotation . |
| Outcome: | The proposed model outperforms previous approaches by a significant margin in QA tasks over text. |
Commonsense Reasoning for Natural Language Processing (2020.acl-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we will outline the various types of commonsense knowledge and discuss techniques to gather and represent commonsence knowledge. |
| Approach: | This tutorial will provide researchers with the critical foundations and recent advances in commonsense representation and reasoning. |
| Outcome: | This tutorial will outline the various types of commonsense and discuss techniques to gather and represent commonsence knowledge while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). |
Natural Logic-guided Autoregressive Multi-hop Document Retrieval for Fact Verification (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent evidence retrieval approaches rely on heuristics and assume hyperlinks between documents. |
| Approach: | They propose a retrieval method that combines a retriever and a proof system that reranks documents and reorders them . |
| Outcome: | The proposed method exceeds or is on par with the current state-of-the-art on FEVER, HoVer and FEVEROUS-S while using 5 to 10 times less memory than competing systems. |