| Challenge: | a recent study examines the "natural" word-order constraints that constrain neural networks . we train models to communicate about paths in a simple gridworld . |
| Approach: | They propose to inoculate a notion of "effort" into neural networks to make their linguistic behavior more human-like. |
| Outcome: | The proposed models show a strong tendency to avoid redundancy and minimize long-distance dependencies. |
Similar Papers
Communication Drives the Emergence of Language Universals in Neural Agents: Evidence from the Word-order/Case-marking Trade-off (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing models of language learning with neural agents lack appropriate cognitive biases in artificial learners. |
| Approach: | They propose a framework where speaking and listening agents learn a miniature language via supervised learning and optimize it for communication via reinforcement learning. |
| Outcome: | The proposed framework replicates the word-order/case-marking trade-off without hard-coding biases in the agents. |
Endowing Neural Language Learners with Human-like Biases: A Case Study on Dependency Length Minimization (2024.lrec-main)
Copied to clipboard
| Challenge: | Comparing the behavior of models with that of human learners can reveal which aspects affect the emergence of this preference. |
| Approach: | They propose to add three factors to the standard neural-agent language learning and communication framework to make the simulation more realistic. |
| Outcome: | The proposed conditions can contribute to a small but significant learning advantage for listeners of verb-initial languages. |
The Effect of Efficient Messaging and Input Variability on Neural-Agent Iterated Language Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have focused on agent-based simulations of language emergence. |
| Approach: | They propose to model the trade-off between word order and inflection in natural languages by using neural network agents. |
| Outcome: | The results show that neural agents strive to maintain the utterance type distribution observed during learning, rather than developing a more efficient or systematic language. |
Do Neural Language Models Overcome Reporting Bias? (2020.coling-main)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained language models can overcome reporting bias by estimating the plausibility of rare but unspoken facts. |
| Approach: | They revisit the experiments conducted by Gordon and Van Durme (2013) . they find that pre-trained language models overestimate the very rare . |
| Outcome: | The proposed approach overestimates the rare at the expense of the rare, while minimizing reporting bias. |
Bias and Fairness in Natural Language Processing (D19-2)
Copied to clipboard
| Challenge: | a tutorial will review the history of bias and fairness studies in machine learning and language processing . |
| Approach: | This tutorial reviews the history of bias and fairness studies in machine learning and language processing . it presents recent community effort to quantify and mitigat bias in natural language processing models . |
| Outcome: | This tutorial reviews the history of bias and fairness studies in machine learning and language processing . it aims to quantify and mitigate bias in natural language processing models for a wide spectrum of tasks . |
Examining the Inductive Bias of Neural Language Models with Artificial Languages (2021.acl-long)
Copied to clipboard
| Challenge: | a novel method for investigating inductive biases of language models using artificial languages is proposed . we show that modern neural architectures used for language modeling are intrinsically black boxes . |
| Approach: | They propose a method to investigate inductive biases of language models using artificial languages . they use languages to create parallel corpora across languages that differ only in word order . |
| Outcome: | The proposed method shows that language models can be used to model a wide variety of languages. |
Priorless Recurrent Networks Learn Curiously (2020.coling-main)
Copied to clipboard
| Challenge: | a recent study shows domain-general recurrent neural networks reproduce human language behaviours . a lack of a unified concept of number agreement between these processes is a limitation of the model . |
| Approach: | They propose to use domain-general recurrent neural networks without explicit linguistic inductive biases to reproduce human language behaviours. |
| Outcome: | The proposed model can learn number agreement within unnatural sentences, the authors show . they show that the model has an effective understanding of singular versus plural for individual sentences . |
End-to-End Bias Mitigation by Modelling Biases in Corpora (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that strong natural language understanding models are prone to relying on unwanted dataset biases without learning the underlying task. |
| Approach: | They propose two learning strategies to train neural models that are more robust to dataset biases and transfer better to out-of-domain datasets. |
| Outcome: | The proposed methods improve robustness in all settings and transfer better to out-of-domain datasets. |
Searching for Structure: Investigating Emergent Communication with Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Human languages have evolved to be structured through repeated language learning and use. |
| Approach: | They propose to use large language models to optimise for implicit biases that shape languages to improve communicative efficiency. |
| Outcome: | The proposed models can be used to study language evolution and open possibilities for human-machine interactions. |
From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models (2023.acl-long)
Copied to clipboard
| Challenge: | Hundreds of studies have highlighted ethical issues in NLP models . |
| Approach: | They propose to measure media biases in LMs trained on diverse data sources . they focus on hate speech and misinformation detection . |
| Outcome: | The proposed methods quantify the fairness of downstream NLP models trained on politically biased LMs. |