Compositional Structured Explanation Generation with Dynamic Modularized Reasoning (2024.starsem-1)
Copied to clipboard
| Challenge: | Large-scale language models have shown remarkable performance on reasoning tasks such as reading comprehension, natural language inference, story generation, etc. |
| Approach: | They propose a compositional structured explanation generation task to test a model's ability to generalize from generating entailment trees to more steps, focusing on the length and shapes of engorgement trees. |
| Outcome: | The proposed model shows competitive compositional generalization abilities in a generation setting. |
Similar Papers
Can Sequence-to-Sequence Transformers Naturally Understand Sequential Instructions? (2023.starsem-1)
Copied to clipboard
| Challenge: | Using a limited annotation budget, we can greatly improve the performance on intermediate steps with a drop in final-step performance. |
| Approach: | They propose to use a pre-supervised sequence-to-sequence transformer to provide training signals on intermediate steps with zero gold supervision instead of only final-step supervision to improve performance. |
| Outcome: | The proposed model significantly improves on intermediate steps with a drop in final-step performance on one subtask, but also shows decreased performance on another subtask. |
Dyna-bAbI: unlocking bAbI’s potential with dynamic synthetic benchmarking (2022.starsem-1)
Copied to clipboard
Ronen Tamari, Kyle Richardson, Noam Kahlon, Aviad Sar-shalom, Nelson F. Liu, Reut Tsarfaty, Dafna Shahaf
| Challenge: | Controlled synthetic tasks are an important resource for diagnosing model behavior. |
| Approach: | They propose a framework that provides fine-grained control over task generation in bAbI. |
| Outcome: | The proposed framework provides fine-grained control over task generation in the bAbI benchmark. |
PDDLEGO: Iterative Planning in Textual Environments (2024.starsem-1)
Copied to clipboard
| Challenge: | Existing methods to plan in textual environments rely on a fully-observed environment where all entity states are known, but are not interpretable. |
| Approach: | They propose to use LLMs to generate a formal representation of the environment that can be solved by a symbolic planner. |
| Outcome: | The proposed model outperforms existing methods in the Coin Collector simulation and Cooking World simulations. |
What do Large Language Models Learn about Scripts? (2022.starsem-1)
Copied to clipboard
| Challenge: | Script Knowledge is important for language understanding but expensive to produce manually and difficult to induce from text due to reporting bias. |
| Approach: | They propose a pipeline-based script induction framework which can generate good quality ESDs for unseen scenarios. |
| Outcome: | The proposed framework produces good quality ESDs for unseen scenarios, but manual evaluation shows there is room for improvement. |
Toward Diverse Precondition Generation (2021.starsem-1)
Copied to clipboard
| Challenge: | a typical goal for language understanding is to logically connect the events of a discourse, but connective events are not described due to their commonsense nature. |
| Approach: | They propose a system that generates unique and diverse preconditions by using an event sampler, candidate generator, and post-processor. |
| Outcome: | The proposed system can generate unique and diverse preconditions without training on diverse examples. |
BiQuAD: Towards QA based on deeper text understanding (2021.starsem-1)
Copied to clipboard
| Challenge: | Recent question answering and machine reading benchmarks require systems to pinpoint the span of the answer to a given text. |
| Approach: | They propose a dataset that requires deeper comprehension to answer questions extractively and deductively. |
| Outcome: | The proposed dataset outperforms existing benchmarks on extractive and deductive questions. |
Fine-tuning BERT with Focus Words for Explanation Regeneration (2020.starsem-1)
Copied to clipboard
| Challenge: | Existing approaches to explain the correct answer in multiple-choice QA are low in F-scores and lack of performance. |
| Approach: | They introduce a lightweight focus feature in a transformer-based NLP model and examine performance improvements. |
| Outcome: | The proposed model achieves the highest scores, second only to a computationally intensive system. |
PipeNet: Question Answering with Semantic Pruning over Knowledge Graphs (2024.starsem-1)
Copied to clipboard
| Challenge: | Existing approaches to utilizing explicit knowledge graphs (KGs) are limited by the number of nodes in the subgraph. |
| Approach: | They propose a grounding-pruning-reasoning pipeline to prune noisy nodes in subgraphs to improve the efficiency of graph reasoning with KG. |
| Outcome: | The proposed method reduces computation cost and memory usage while obtaining decent representation of pruned subgraphs. |
DeepA2: A Modular Framework for Deep Argument Analysis with Pretrained Neural Text2Text Language Models (2022.starsem-1)
Copied to clipboard
| Challenge: | ArgumentAnalyst is a multi-dimensional, modular framework for performing deep argument analysis using existing pre-trained language models (PTLMs). |
| Approach: | They propose a multi-dimensional, modular framework for performing deep argument analysis using current pre-trained language models (PTLMs) ArgumentAnalyst reconstructs argumentative texts as valid arguments by inserting missing premises and conclusions, formalizing inferences, and coherently linking the reconstruction to the source text. |
| Outcome: | The proposed framework can replicate established heuristics, cope with plurality of correct solutions, and exploit higher-order evidence. |
Compositional generalization with a broad-coverage semantic parser (2022.starsem-1)
Copied to clipboard
| Challenge: | Recent work has shown that compositional generalization on COGS is difficult and complex. |
| Approach: | They propose a compositional semantic parser that solves compositional generalization on COGS dataset. |
| Outcome: | The AM parser solves compositional generalization on the COGS dataset. |