FLIN: A Flexible Natural Language Interface for Web Navigation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing semantic parsing and slot-filling techniques cannot adapt to many different websites without being constantly re-trained. |
| Approach: | They propose a natural language interface for web navigation that maps user commands to concept-level actions rather than low-level UI actions. |
| Outcome: | The proposed interface can adapt to new websites in a given domain. |
Similar Papers
Mapping natural language commands to web elements (D18-1)
Copied to clipboard
| Challenge: | a dataset of over 50,000 natural language commands captures various phenomena, including functional references, relational reasoning, and visual reasoning. |
| Approach: | They propose a task that requires the user to choose the correct element on a web page . they use a dataset of over 50,000 natural language commands to map these to web pages . |
| Outcome: | The proposed task can be viewed as a reference game based on a dataset of over 50,000 natural language commands . |
Grounding Open-Domain Instructions to Automate Web Support Tasks (2021.naacl-main)
Copied to clipboard
| Challenge: | RUSS is a task and dataset to ground natural language instructions on the web to perform previously unseen tasks. |
| Approach: | They build a task and dataset to ground AI agents from open-domain, step-by-step instructions on the web. |
| Outcome: | The proposed model outperforms existing models that map instructions to actions without WebLang. |
Commonsense Reasoning for Natural Language Processing (2020.acl-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we will outline the various types of commonsense knowledge and discuss techniques to gather and represent commonsence knowledge. |
| Approach: | This tutorial will provide researchers with the critical foundations and recent advances in commonsense representation and reasoning. |
| Outcome: | This tutorial will outline the various types of commonsense and discuss techniques to gather and represent commonsence knowledge while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). |
FLOR: On the Effectiveness of Language Adaptation (2024.lrec-main)
Copied to clipboard
Severino Da Dalt, Joan Llop, Irene Baucells, Marc Pamies, Yishi Xu, Aitor Gonzalez-Agirre, Marta Villegas
| Challenge: | Large language models have amply proven their capabilities, but low- and mid-resource languages do not have access to the necessary means to train such models from scratch. |
| Approach: | They use a 26B tokens corpus to further pre-train BLOOM, giving rise to FLOR models. |
| Outcome: | The proposed model achieves consistent gains across Catalan and Spanish tasks. |
TransBench: Breaking Barriers for Transferable Graphical User Interface Agents in Dynamic Digital Environments (2025.findings-acl)
Copied to clipboard
Yuheng Lu, Qian Yu, Hongru Wang, Zeming Liu, Wei Su, Yanping Liu, Yuhang Guo, Maocheng Liang, Yunhong Wang, Haifeng Wang
| Challenge: | Existing GUI agents struggle to adapt to dynamic and interconnected nature of real-world digital environments, authors show . |
| Approach: | They propose a benchmark to evaluate the transferability of GUI agents across three key dimensions . transBench includes 15 app categories with diverse functionalities . |
| Outcome: | The proposed benchmark shows that existing GUI agents struggle to adapt to dynamic, interconnected environments. |
Beyond Browsing: API-Based Web Agents (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing web agents use browsers to facilitate human activities such as online shopping, online planning, and other work-related tasks. |
| Approach: | They propose to use web browsers as an interface to interact with online content through application programming interfaces (APIs) they propose to call APIs and use Hybrid Agents to perform online tasks. |
| Outcome: | The proposed agents outperform web Browsing Agents on a widely-used and realistic benchmark for web navigation tasks. |
Adaptor: Objective-Centric Adaptation Framework for Language Models (2022.acl-demo)
Copied to clipboard
| Challenge: | Adaptor library aims to simplify complex training processes requiring customizations. |
| Approach: | They introduce Adaptor library which transposes traditional model-centric approach to objective-centric training pipeline with Objective as central abstraction. |
| Outcome: | The proposed framework simplifies training processes and improves reproducibility. |
FITAnnotator: A Flexible and Intelligent Text Annotation System (2021.naacl-demos)
Copied to clipboard
| Challenge: | In this paper, we introduce FITAnnotator, a generic web-based tool for efficient text annotation. |
| Approach: | They propose a generic web-based tool for efficient text annotation. |
| Outcome: | The proposed tool is based on a fully modular architecture and provides three kinds of interfaces to annotate instances, evaluate annotation quality and manage the annotation task for annotators, reviewers and managers. |
Interactive Plot Manipulation using Natural Language (2021.naacl-demos)
Copied to clipboard
| Challenge: | a new interactive plotting agent is available for programming with natural language . the interactive aspect allows users to manipulate plots using natural language instructions. |
| Approach: | They propose an interactive natural language interface for plotting that maps language to plot updates. |
| Outcome: | The proposed system maps language to plot updates within an interactive programming environment. |
WebWISE: Unlocking Web Interface Control for LLMs via Sequential Exploration (2024.findings-naacl)
Copied to clipboard
| Challenge: | Prior work to control software has used reinforcement learning (RL), requiring many demonstrations and trials to learn simple interaction tasks. |
| Approach: | They propose a Large Language Model to automatically perform web software tasks using click, scroll, and text in- put operations using filtered Document Object Models as observations. |
| Outcome: | The proposed method performs better on the MiniWob++ benchmark with only one in-context example. |