DANLI: Deliberative Agent for Following Natural Language Instructions (2022.emnlp-main)
Copied to clipboard
Yichi Zhang, Jianing Yang, Jiayi Pan, Shane Storks, Nikhil Devraj, Ziqiao Ma, Keunwoo Yu, Yuwei Bao, Joyce Chai
| Challenge: | Recent work on embodied AI agents that can perform tasks by following human language instructions is limited by reactive methods, which are insufficient for long-horizon complex tasks. |
| Approach: | They propose a neuro-symbolic deliberative agent that, while following language instructions, proactively applies reasoning and planning based on its neural and symbolic representations acquired from past experience. |
| Outcome: | The proposed agent achieves greater than 70% improvement over reactive baselines on the challenging TEACh benchmark. |
Similar Papers
tagE: Enabling an Embodied Agent to Understand Human Instructions (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing systems for natural language understanding (NLU) are limited due to the inherent ambiguity and incompleteness inherent in natural language. |
| Approach: | They propose a system to extract tasks from natural language instructions and map them to robots' established collection of skills. |
| Outcome: | The proposed system outperforms baseline models in the training and evaluation of a dataset featuring complex instructions. |
OPEx: A Component-Wise Analysis of LLM-Centric Agents in Embodied Instruction Following (2024.acl-long)
Copied to clipboard
| Challenge: | Embodied Instruction Following (EIF) is a crucial task in embodied learning . however, there is n'a unified understanding regarding the impact of various components on task performance . |
| Approach: | They propose a framework that delineates the core components essential for embodied learning tasks . they integrate a multi-agent design into the Planner component of their LLM-centric architecture . |
| Outcome: | OPEx delineates the core components essential for solving embodied learning tasks . integrating a multi-agent design into the Planner component of the LLM-centric architecture further elevates performance. |
D-Artemis: A Deliberative Cognitive Framework for Mobile GUI Multi-Agents (2026.findings-acl)
Copied to clipboard
Hongze Mi, Yibo Feng, WenJie Lu, Yuqi Wang, Jinyuan Li, Song Cao, He Cui, Tengfei Tian, Xuelin Zhang, Haotian Luo, Di Sun, Jun Fang, Hua Chai, Naiqiang Tan, Gang Pan
| Challenge: | Graphical User Interface (GUI) agents aim to automate a wide spectrum of human tasks by emulating user interaction. |
| Approach: | They propose a deliberative framework that leverages a fine-grained tip retrieval mechanism to inform its decision-making process. |
| Outcome: | The proposed framework achieves SOTA among open-source general models on AndroidWorld and ScreenSpot-V2 . it leverages a fine-grained, app-specific tip retrieval mechanism to inform its decision-making process . |
Neuro-Symbolic Agentic Reinforcement Learning for Long-Term Original Character Companionship and Interaction (2026.acl-short)
Copied to clipboard
| Challenge: | Existing LLM-based agents that are optimized by prompting or supervised fine-tuning exhibit a generalization gap in long-horizon, socially rich interactions. |
| Approach: | They propose a framework that formalizes OC companion agents’ interactions as a POMDP and decomposes the agent into three sub-policies optimized via closed-loop RL from AI feedback with verifiable rewards in a graph-constrained action space. |
| Outcome: | The proposed framework formalizes OC companion agents’ interactions as a POMDP and decomposes the agent into three sub-policies (Router, Memory, and Persona) with verifiable rewards in a graph-constrained action space. |
Neuro-Symbolic Natural Language Processing (2025.emnlp-tutorials)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have limitations in terms of safe and controlled reasoning, interpretability and adaptability . this tutorial aims to bridge the gap between the practical performance of LLMs and the principled modelling of language and inference of formal methods. |
| Approach: | This tutorial aims to bridge the gap between the practical performance of Large Language Models and the principled modelling of language and inference of formal methods. |
| Outcome: | This tutorial aims to bridge the gap between the performance of LLMs and the principled modelling of language and inference of formal methods. |
Model-Based Imaginative Planning for Embodied Agents (2026.acl-long)
Copied to clipboard
Junru Song, Hengzhe Jin, Yucong Huang, Tingsong Jiang, Weien Zhou, Feifei Wang, Yang Yang, Ying Wen, Wen Yao
| Challenge: | a lightweight world model converts raw pixels into object-centric symbolic states amenable to language-based reasoning . IMPLEMENT is a framework for grounding language agents in visual embodied environments . |
| Approach: | They propose a model-based reasoning framework that enables frozen large language models to perform imaginative planning. |
| Outcome: | The proposed framework can be used to ground language agents in visual embodied environments. |
A Language-First Approach for Procedure Planning (2023.findings-acl)
Copied to clipboard
| Challenge: | Developing intelligent agents requires the ability to produce plans on the fly based on visual observations. |
| Approach: | They propose a language-first procedure planning framework with a modularized design . they first align current and goal observations with corresponding steps and then use a pre-trained LM to predict intermediate steps. |
| Outcome: | The proposed framework matches state-of-the-art procedures on COIN and CrossTask benchmarks. |
ALFRED-L: Investigating the Role of Language for Action Learning in Interactive Visual Environments (2022.emnlp-main)
Copied to clipboard
Arjun Akula, Spandana Gella, Aishwarya Padmakumar, Mahdi Namazifar, Mohit Bansal, Jesse Thomason, Dilek Hakkani-Tur
| Challenge: | Embodied Vision and Language Task Completion requires an embodied agent to interpret natural language instructions and egocentric visual observations to navigate through and interact with environments. |
| Approach: | They propose to use ALFRED to test whether models can adapt to tasks not seen during training that require the same types of language understanding as ALFred. |
| Outcome: | The proposed model can adapt to tasks that require the same types of language understanding as ALFRED-L. |
Teaching Embodied Reinforcement Learning Agents: Informativeness and Diversity of Language Use (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for embodied agents to learn and perform tasks use low-level instructions, which may not reflect natural human communication. |
| Approach: | They propose to use different types of language inputs to facilitate reinforcement learning (RL) embodied agents. |
| Outcome: | The proposed methods show that agents trained with diverse and informative language can achieve enhanced generalization and fast adaptation to new tasks in an open world. |
MADRA: Multi-Agent Debate for Risk-Aware Embodied Planning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing safety alignment methods, such as RLHF, fall into a Safety-Utility Trade-off, resulting in severe over-rejection of benign household instructions. |
| Approach: | They propose a meta-cognitive Critical Agent that evaluates peer debates using a structured argumentation framework derived from the Toulmin Model. |
| Outcome: | The proposed architecture outperforms existing systems in the SafeAware-VH benchmark. |