Compositional Data and Task Augmentation for Instruction Following (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models that understand spatial concepts and compositional language are inadequate for executing natural language instructions in a physically grounded domain. |
| Approach: | They propose to use knowledge-free auxiliary signals to help the model understand compositional instructions and provide supervision for the instruction's components. |
| Outcome: | The proposed model correctly identifies the source block while the existing model fails on this example. |
Similar Papers
Generalization in Instruction Following Systems (2021.naacl-main)
Copied to clipboard
| Challenge: | Understanding and executing natural language instructions in a grounded domain is one of the hallmarks of artificial intelligence. |
| Approach: | They propose a learning strategy that involves data augmentation to improve the model's performance. |
| Outcome: | The proposed learning strategy outperforms state-of-the-art models in the blocks world domain while satisfying our expectations much better. |
Improving Natural Language Interaction with Robots Using Advice (N19-1)
Copied to clipboard
| Challenge: | Recent studies focus on learning models for physically grounded language understanding tasks such as the blocks world domain. |
| Approach: | They propose a protocol for including advice, high-level observations about the task, which can help constrain the agent’s prediction. |
| Outcome: | The proposed approach can be extended to include advice, high-level observations about the task, and reduce the effort involved in supplying the advice. |
How Many Data Samples is an Additional Instruction Worth? (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent introduced instruction-paradigm empowers non-expert users to leverage NLP resources by defining a new task in natural language. |
| Approach: | They propose to define a task in natural language without creating task-specific datasets or building models. |
| Outcome: | The proposed model outperforms multitask learning models but is far from state-of-the-art task-specific models. |
Compositional Generalization in Grounded Language Learning via Induced Model Sparsity (2022.naacl-srw)
Copied to clipboard
| Challenge: | induced model sparsity can help achieve compositional generalization and sample efficiency in grounded language learning problems. |
| Approach: | They propose a model that encourages sparse correlations between words and attributes to find a goal in a language-conditioned navigation problem with disentangled observations. |
| Outcome: | The proposed agent maintains high performance even when learning from a handful of demonstrations. |
Learning with Latent Language (N18-1)
Copied to clipboard
| Challenge: | Using the space of natural language strings as a parameter space is an effective way to capture natural task structure. |
| Approach: | They propose to use natural language as a parameter space for few-shot learning problems including classification, transduction and policy search. |
| Outcome: | The proposed model outperforms models with a linguistic parameterization on image classification, text editing, and reinforcement learning. |
Sub-Instruction Aware Vision-and-Language Navigation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Despite significant advances, few previous works are able to fully utilize the strong correspondence between visual and textual sequences. |
| Approach: | They propose to provide agents with fine-grained annotations during training and provide them with sub-instructions and their corresponding paths. |
| Outcome: | The proposed method improves the performance of four state-of-the-art agents in a room-to-room (R2R) benchmark dataset. |
LLM-driven Instruction Following: Progresses and Concerns (2023.emnlp-tutorial)
Copied to clipboard
| Challenge: | a tutorial on task instruction is aimed at researchers and practitioners interested in NLP generalization . labeled examples are unlikely to be available in large numbers or do not exist . |
| Approach: | This tutorial will examine the progress of natural language processing (NLP) using labeled examples. authors propose that task instructions act as a novel resource for supervision. |
| Outcome: | This tutorial aims to answer questions about instruction-driven NLP . it focuses on the use of task instructions in a low-shot scenario . |
Compositional Generalization with Grounded Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for combining language models with knowledge graphs struggle with generalization to sequences of unseen lengths and novel combinations of seen base components. |
| Approach: | They propose a procedure for generating natural language questions paired with knowledge graphs that targets different aspects of compositionality and avoids grounding models in information already encoded in their weights. |
| Outcome: | The proposed method fails to generalize to unseen lengths and to novel combinations of seen base components. |
SplitThenMerge: Token-Level Skill-Compositional Sparse Mixture-of-Experts for Complex Domain-Specific Tasks (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing domain adaptation methods train heterogeneous skills together, making it difficult to reliably coordinate multiple skills when solving complex tasks. |
| Approach: | They propose a framework that decomposes domain competence into atomic skills and composes them dynamically during generation. |
| Outcome: | The proposed framework decomposes domain competence into atomic skills, trains them independently, and composes them dynamically during generation. |
Learning to Perform Complex Tasks through Compositional Fine-Tuning of Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on how to encode compositional task structure has been limited by semantic parsing and multihop reasoning for the purpose of Q&A. |
| Approach: | They propose an approach to decomposing a target task into component tasks and fine-tuning smaller LMs on a curriculum of such component tasks. |
| Outcome: | The proposed approach outperforms end-to-end learning even with equal data, and gets better as more component tasks are modeled. |