Situated Mapping of Sequential Instructions to Actions with Single-step Reward Observation (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to map context-dependent sequential instructions to actions are based on discourse and state dependencies . we evaluate on SCONE domains and show absolute accuracy improvements of 9.8%-25.3% . |
| Approach: | They propose a model that considers previous utterances and the state of the world to map sequential instructions to actions. |
| Outcome: | The proposed model improves on the SCONE domains and on the target domains. |
Similar Papers
Semi-Supervised Dialogue Policy Learning via Stochastic Reward Estimation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for dialogue policy optimization do not provide sufficient supervision signals at the end of dialogues. |
| Approach: | They propose to learn from state-action pairs of an optimal policy to provide turn-by-turn rewards. |
| Outcome: | The proposed approach outperforms competitive policy learning baselines on a benchmark multi-domain dataset. |
Executing Instructions in Situated Collaborative Interactions (D19-1)
Copied to clipboard
Alane Suhr, Claudia Yan, Jack Schluger, Stanley Yu, Hadi Khader, Marwa Mouallem, Iris Zhang, Yoav Artzi
| Challenge: | a collaborative game with natural language instruction allows users to adapt to the system abilities by changing their language or deciding to accomplish tasks themselves. |
| Approach: | They propose a collaborative game where a user instructs a system to complete tasks, but acts alongside it. |
| Outcome: | The proposed game allows users to adapt to the system abilities by changing their language or deciding to accomplish tasks themselves. |
Large-scale Lifelong Learning of In-context Instructions and How to Tackle It (2023.acl-long)
Copied to clipboard
| Challenge: | In-context instruction learning is a method to improve the target PLM’s instance- and task-level generalization performance as it observes more tasks. |
| Approach: | They propose to fine-tune a Pre-trained Language Model (PLM) on a set of tasks with in-context instructions and to extend this property to a scenario in which tasks are fed to the target PLM in a sequential manner. |
| Outcome: | The proposed method achieves noticeable improvements in both types of generalization, nearly reaching the upper bound performance obtained through joint training. |
Continual Learning for Grounded Instruction Generation by Observing Human Following Behavior (2021.tacl-1)
Copied to clipboard
| Challenge: | Despite its potential and prevalence, this signal is understudied for learning to generate natural language. |
| Approach: | They propose to use this signal to improve the system's ability to generate instructions via contextual bandit learning. |
| Outcome: | The proposed system improves its ability to generate natural language through interaction with users, and the results are shown. |
Mapping Instructions to Actions in 3D Environments with Visual Goal Prediction (D18-1)
Copied to clipboard
| Challenge: | Existing models that map from inputs to actions are inefficient and require hand-crafted meaning representations. |
| Approach: | They propose to decompose instruction execution to goal prediction and action generation . they introduce two benchmarks for instruction following: LANI and CHAI . |
| Outcome: | The proposed model decomposes instruction execution to goal prediction and action generation. |
Fine-Tuning Large Language Models with Sequential Instructions (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing instruction-tuned models struggle to adhere to a query with multiple intentions, which impairs their performance when the completion of several tasks is demanded by a single command. |
| Approach: | They develop an automatic process that turns existing data into diverse and complex task chains and a new benchmark to evaluate a model’s ability to follow all the instructions in a sequence. |
| Outcome: | The proposed model can follow instructions better and deliver higher results in coding, maths, and open-ended generation. |
Inferring Rewards from Language in Context (2022.acl-long)
Copied to clipboard
| Challenge: | a new study grounding language to reward functions extends the standard instruction following setup in this way. |
| Approach: | They propose a model that infers rewards from language pragmatically by reasoning about how speakers choose utterances to elicit desired actions and reveal information about their preferences. |
| Outcome: | The proposed model infers rewards from language pragmatically on a flight–booking task with natural language. |
Pre-Learning Environment Representations for Data-Efficient Neural Instruction Following (P19-1)
Copied to clipboard
| Challenge: | Using logical forms, neural networks can sometimes require orders of magnitude more data to map from natural language instructions to state transitions (actions) |
| Approach: | They propose to map from natural language instructions to state transitions (actions) they augment a baseline learner with an initial environment-learning phase that uses observations of language-free state transition to induce a suitable latent representation of actions before processing the instruction-following training data. |
| Outcome: | The proposed model improves performance over systems whose representations are learned from limited instructional data alone. |
Severing the Edge Between Before and After: Neural Architectures for Temporal Ordering of Events (2020.emnlp-main)
Copied to clipboard
Miguel Ballesteros, Rishita Anubhai, Shuai Wang, Nima Pourdamghani, Yogarshi Vyas, Jie Ma, Parminder Bhatia, Kathleen McKeown, Yaser Al-Onaizan
| Challenge: | Existing models for temporal ordering of events rely on pretrained representations, transfer and multitask learning, and self-training techniques. |
| Approach: | They propose a neural architecture and a set of training methods for ordering events by predicting temporal relations by pre-training models. |
| Outcome: | The proposed models can predict temporal relations between two pairs of events within a span of text and identify temporal relationships between them. |
Sequence-to-Sequence Learning for Task-oriented Dialogue with Dialogue State Representation (C18-1)
Copied to clipboard
| Challenge: | Existing pipeline models for task-oriented dialogue system require explicit modeling of dialogue states and hand-crafted action spaces to query domain-specific knowledge base. |
| Approach: | They propose a framework that leverages the advantages of classic pipeline and sequence-to-sequence models. |
| Outcome: | The proposed framework outperforms baseline models on automatic and human evaluation on a Stanford Multi-turn Multi-domain task-oriented dialogue dataset. |