When Language Models Fall in Love: Animacy Processing in Transformer Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Animacy is not always expressed directly in language, but it manifests indirectly in English . atypically animate entities are easier to remember and prioritized in visual processing . |
| Approach: | They find that LMs behave much like humans when presented with entities whose animacy is typical. |
| Outcome: | The proposed model can learn about animacy even when presented with atypically animate entities. |
Similar Papers
Living Machines: A study of atypical animacy (2020.coling-main)
Copied to clipboard
Mariona Coll Ardanuy, Federico Nanni, Kaspar Beelen, Kasra Hosseini, Ruth Ahnert, Jon Lawrence, Katherine McDonough, Giorgia Tolfo, Daniel CS Wilson, Barbara McGillivray
| Challenge: | atypical animacy is the property of being alive, but discrepancies are not uncommon . a typical animate is represented as either animate or inanimate in a text . |
| Approach: | They propose a method for determining whether an entity is represented as animate in a text . they use a nineteenth-century English text to analyze animacy . |
| Outcome: | The proposed method improves on an established animacy dataset and a newly introduced resource. |
The Cross-linguistic Role of Animacy in Grammar Structures (2025.acl-long)
Copied to clipboard
| Challenge: | Animacy is a semantic feature of nominals and follows a hierarchy of personal pronouns . it is argued that soft tendencies may be the cause of animacy constraints . however, there is no empirical evidence for this . |
| Approach: | They propose a method to reliably classify animacy classes of nominals in 11 languages from 5 families using multilingual large language models and word sense disambiguation datasets. |
| Outcome: | The proposed method shows that animacy displays consistent cross-linguistic tendencies in terms of preferred morphosyntactic constructions, but not always in line with received wisdom. |
The Morpho-syntactic Annotation of Animacy for a Dependency Parser (L18-1)
Copied to clipboard
| Challenge: | Animacy is a feature found in nouns such as 'gender', 'number' and 'case' that improves parser accuracy. |
| Approach: | They propose an annotation scheme and parser results for the animacy feature in Russian and Arabic, morphologically rich languages, using the universal dependency framework. |
| Outcome: | The proposed scheme and parser improve on the animacy feature in Russian and Arabic, and the results show that the feature is more accurate than other features found in nouns, namely, 'gender', , and 'number' |
Are Transformers a Modern Version of ELIZA? Observations on French Object Verb Agreement (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that unsupervised sentence representations of neural networks encode syntactic information by observing that neural language models are able to predict the agreement between a verb and its subject. |
| Approach: | They propose to take an alternative look at these results by studying whether neural networks are able to build an abstract sentence representation rather than capture surface statistical regularities. |
| Outcome: | The proposed model can achieve high accuracy on the long-range French object-verb agreement, indicating a possible flaw in the model's syntactic ability. |
Transformers Can Represent n-gram Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing work studies the transformer architecture in terms of language acceptance . n-gram LMs can represent any ngram lm, giving a concrete lower bound on their probabilistic representational capacity. |
| Approach: | They propose to use transformer LMs to represent n-gram LM, a class of language models . they show transformer lms can exactly represent any ngram lmm, giving them a lower bound . |
| Outcome: | The proposed model can represent any n-gram LM, giving a lower bound on their probabilistic representational capacity. |
What Context Features Can Transformer Language Models Use? (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that transformer-based language models benefit from conditioning on contexts of hundreds to thousands of previous tokens. |
| Approach: | They propose to use lexical and structural information to ablate usable information in transformer language models. |
| Outcome: | The proposed model improves when conditioning on contexts of thousands of previous tokens. |
How Do Language Models Acquire Character-Level Information? (2026.eacl-long)
Copied to clipboard
| Challenge: | Language models (LMs) implicitly encode character-level information, despite not being explicitly provided during training. |
| Approach: | They analyze how language models acquire character-level knowledge by comparing them to standard settings. |
| Outcome: | The results show that LMs do not treat words as opaque tokens, but instead treat them as tokens. |
Visual Grounding Helps Learn Word Meanings in Low-Data Regimes (2024.naacl-long)
Copied to clipboard
| Challenge: | Modern neural language models (LMs) require distinctly un-human-like ways to achieve these results. |
| Approach: | They train a diverse set of LM architectures with and without auxiliary visual supervision on datasets of varying scales. |
| Outcome: | The proposed models exhibit better learning of syntactic categories, lexical relations, semantic features, word similarity and alignment with human neural representations. |
A New Approach to Animacy Detection (C18-1)
Copied to clipboard
| Challenge: | Animacy is a property for a referent to be an agent, and prior work has classified words as either animate or inanimate. |
| Approach: | They propose a method that uses supervised machine learning and hand-built rules to classify the animacy of co-reference chains. |
| Outcome: | The proposed method achieves state-of-the-art performance on a 142-text dataset . it leverages word embeddings over referring expressions, parts of speech, and grammatical and semantic roles . |
Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Unlike literal expressions, idioms’ meanings do not follow from their parts, posing a challenge for neural machine translation (NMT). |
| Approach: | They examine the mechanics of the dominant NMT model, Transformer, and their effect on their understanding of idioms. |
| Outcome: | The proposed model over-generates compositional, literal translations and is unable to translate idioms accurately. |