Wanjun Zhong, Siyuan Wang, Duyu Tang, Zenan Xu, Daya Guo, Yining Chen, Jiahai Wang, Jian Yin, Ming Zhou, Nan Duan
| Challenge: | Existing models with implicit reasoning ability struggle to solve analytical reasoning of text. |
| Approach: | They propose an approach to analyze text and use it to perform reasoning over it. |
| Outcome: | The proposed approach outperforms pre-trained models on an analysis of the Law School Admission Test dataset. |
Similar Papers
Complex Reasoning in Natural Language (2023.acl-tutorials)
Copied to clipboard
| Challenge: | Recent research shows that pretrained language models are often brittle for complex reasoning tasks. |
| Approach: | They propose to use pre-trained language models to teach machines to reason over texts . they will review recent promising approaches to tackling complex reasoning tasks . |
| Outcome: | This tutorial reviews promising approaches to complex reasoning tasks . it reviews the methods that can be used to augment models with robustness . |
How Well Do Text Embedding Models Understand Syntax? (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing text embedding models have not addressed syntactic understanding challenges, highlighting ineffectiveness and enhancing generalization ability. |
| Approach: | They propose to examine the ability of text embedding models to generalize across syntactic contexts. |
| Outcome: | The proposed models exhibit high similarity socres at this simple task. |
Evaluating the Rationale Understanding of Critical Reasoning in Logical Reading Comprehension (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent large language models (e.g., InstructGPT) struggle to answer subquestions written for the incorrect options of the main questions, implying that they have a limited capability for explaining why incorrect alternatives should be eliminated. |
| Approach: | They crowdsource free-form rationale for main questions and generate subquestion texts that explain why we should select or eliminate answer options. |
| Outcome: | The proposed dataset shows that large language models struggle to answer the subquestions even if they are able to answer main questions correctly. |
Scalable Construction and Reasoning of Massive Knowledge Bases (N18-6)
Copied to clipboard
| Challenge: | Existing knowledge mining systems assume abundant human annotations for training high quality machine learning models, which is impractical when trying to deploy IE systems to a broad range of domains, settings and languages. |
| Approach: | They introduce how to extract structured facts from text corpora to construct knowledge bases. |
| Outcome: | The proposed methods are weakly-supervised and domain-independent for knowledge base construction across various domains. |
Can AMR Assist Legal and Logical Reasoning? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) has been shown to be useful for many downstream tasks. |
| Approach: | They propose neural architectures that utilize linearised AMR graphs in combination with pre-trained language models to capture logical relationships on multiple choice question answering tasks. |
| Outcome: | The proposed models outperform text-only baselines but outperformed text models, suggesting complementary abilities. |
The Knowledge Microscope: Features as Better Analytical Lenses than Neurons (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that features are superior analytical units for understanding factual knowledge in Language Models. |
| Approach: | They propose a feature-based editing method that decomposes neurons into features rather than neurons to understand the mechanisms of factual knowledge in Language Models. |
| Outcome: | The proposed method demonstrates superior performance over neuron-based approaches in erasing privacy-sensitive information from LMs. |
From Heuristic to Analytic: Cognitively Motivated Strategies for Coherent Physical Commonsense Reasoning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models have shown impressive performance in various language tasks, but are prone to spurious correlations and illusory information. |
| Approach: | They propose to use pre-trained language models to justify decisions with formalized, coherent reasoning chains. |
| Outcome: | The proposed strategies improve coherence of rationalizations yielding state-of-the-art results on Tiered Reasoning for Intuitive Physics (TRIP). |
The Argument Reasoning Comprehension Task: Identification and Reconstruction of Implicit Warrants (N18-1)
Copied to clipboard
| Challenge: | Existing methods for analyzing warrants in natural language arguments are insufficient. |
| Approach: | They propose a method for reconstructing warrants from news comments . they use a crowdsourcing process to obtain warrants for 2k authentic arguments . |
| Outcome: | The proposed method will define a substantial step towards automatic warrant reconstruction. |
In-Context Analogical Reasoning with Pre-Trained Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Analogical reasoning is a fundamental capacity of human cognition that allows us to reason abstractly about novel situations by relating them to past experiences. |
| Approach: | They apply large pre-trained language models to visual Raven’s Progressive Matrices (RPM) and use language-based abstractions to support analogy in AI systems. |
| Outcome: | The proposed language-based abstractions outperform human models on Raven’s Progressive Matrices and supervised vision-based methods. |
Towards Reasoning in Large Language Models: A Survey (2023.findings-acl)
Copied to clipboard
| Challenge: | Reasoning is a fundamental aspect of human intelligence that plays a crucial role in many intellectual activities. |
| Approach: | They propose to improve LLMs' ability to elicit reasoning by providing exemplars or prompts to model reasoning. |
| Outcome: | This paper provides a comprehensive overview of the state of knowledge on reasoning in large language models. |