How agents see things: On visual representations in an emergent language game (D18-1)
Copied to clipboard
| Challenge: | Existing studies focus on the agents’ symbol usage, rather than on their representation of visual input. |
| Approach: | They propose to use visual representations of objects to create language-like communication systems by integrating them with the visual input of a game. |
| Outcome: | The proposed model and setup of Lazaridou et al. (2017) show that the representations of the agents' symbols do not capture the conceptual properties of the objects depicted in the input images. |
Similar Papers
The Emergence of Compositional Languages in Multi-entity Referential Games: from Image to Graph Representations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Language Emergence research uses jointly trained artificial agents to solve a task. |
| Approach: | They propose a multi-entity game in which targets include multiple entities that are spatially related. |
| Outcome: | The proposed multi-entity game shows that the emergent languages exhibit a considerable degree of compositionality, but not over all features. |
So many design choices: Improving and interpreting neural agent communication in signaling games (2023.findings-acl)
Copied to clipboard
| Challenge: | Emergent language games are experimental protocols designed to model how communication may arise among a group of agents. |
| Approach: | They propose to adopt a signaling game in which a sender is exposed to an image and generates a sequence of symbols that is transmitted to a receiver. |
| Outcome: | The proposed language improves when the sender is exposed to an image and generates a sequence of symbols that is transmitted to a receiver. |
Context Shapes Emergent Communication about Concepts at Different Levels of Abstraction (2024.lrec-main)
Copied to clipboard
| Challenge: | Concept-level reference game allows speakers to communicate concepts at different levels of abstraction and in different contexts. |
| Approach: | They use a symbolic dataset that disentangles concept type and context to study the influence of these factors on the emerging language. |
| Outcome: | The proposed model disentangles concept type and context to study the communication of concepts at different levels of abstraction and in different contexts. |
Emergent Linguistic Phenomena in Multi-Agent Communication Games (D19-1)
Copied to clipboard
| Challenge: | a recent study examines the behavior of linguistic agents in a community-level setting . a linguistic continuum emerges where neighboring languages are more mutually intelligible than farther removed ones . |
| Approach: | They propose a multi-agent communication framework for studying linguistic phenomena at the community level. |
| Outcome: | The proposed framework can reproduce complex linguistic behavior observed in natural language . it can be used to study interactions between perceptually-enabled agents . |
Emergence of Hierarchical Reference Systems in Multi-agent Communication (2022.coling-1)
Copied to clipboard
| Challenge: | a hierarchical reference system allows the selection of the most appropriate level of specificity for a given context. |
| Approach: | They propose a hierarchical reference game to study the emergence of hierarchic reference systems in artificial agents. |
| Outcome: | The proposed game shows that agents can generalize to new concepts . the hierarchical reference game is based on a simplified world . |
Co-evolution of language and agents in referential games (2021.eacl-main)
Copied to clipboard
| Challenge: | Referential games allow neural agents to learn language, but they do not take into account the learning biases of the learners. |
| Approach: | They propose to model cultural and architectural evolution in a population of agents to take into account learning biases of the language learners and let them co-evolve. |
| Outcome: | The proposed model outperforms cultural transmission in a population of agents and takes into account learning biases of the learners. |
Reading and Acting while Blindfolded: The Need for Semantics in Text Game Agents (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work has used text-based games as a testbed for developing autonomous agents that operate using natural language. |
| Approach: | They propose an inverse dynamics decoder to regularize representation space and encourage exploration to reduce the amount of semantic information available to a learning agent. |
| Outcome: | The proposed model achieves high scores even in the absence of language semantics on Zork I . |
Countering Language Drift via Visual Grounding (D19-1)
Copied to clipboard
| Challenge: | Emergent multi-agent communication protocols are different from natural language . a long-standing goal of artificial intelligence research is to develop agents that can cooperate with other agents . |
| Approach: | They propose to use syntactic and semantic constraints to improve communication . they propose to combine these constraints with auxiliary training constraints to reduce language drift . |
| Outcome: | a new study shows that pre-trained agents retain English syntax while learning to convey intended meaning . the proposed training constraints can be used to mitigate language drift . |
Emergent Language-Based Coordination In Deep Multi-Agent Systems (2022.emnlp-tutorials)
Copied to clipboard
| Challenge: | Pre-trained deep networks are the standard building blocks of modern AI applications. |
| Approach: | This tutorial will introduce deep net emergent communication and discuss current shortcomings . participants will implement and analyze two emergentic communication setups from the literature . |
| Outcome: | The presentation will cover various topics from the present and recent past, as well as discussing current shortcomings and suggest future directions. |
Connecting Language and Vision to Actions (P18-5)
Copied to clipboard
| Challenge: | Recent advances in language and vision have made incredible progress in describing images and interacting with visual content in a physical or embodied environment. |
| Approach: | This tutorial will provide an overview of the growing number of multimodal tasks and datasets that combine textual and visual understanding. |
| Outcome: | This tutorial will review the state-of-the-art approaches to selected tasks such as image captioning, visual question answering and visual dialog. |