Dependency resolution at the syntax-semantics interface: psycholinguistic and computational insights on control dependencies (2023.acl-long)
Copied to clipboard
| Challenge: | Using psycholinguistic and computational experiments, we compare the ability of humans and several pre-trained masked language models to correctly identify control dependencies in Spanish sentences. |
| Approach: | They compare the ability of humans and several pre-trained masked language models to correctly identify control dependencies in Spanish sentences such as ‘José le prometió/ordenó a Mara ser ordenado/a’. |
| Outcome: | The models fail to identify the correct antecedent in non-adjacent dependencies, showing their reliance on linearity. |
Similar Papers
A Psycholinguistic Evaluation of Language Models’ Sensitivity to Argument Roles (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a systematic evaluation of large language models' sensitivity to argument roles is presented . a recent study shows that argument roles have a delayed impact on verb prediction in human sentence processing. |
| Approach: | They propose to replicate psycholinguistic studies on human argument role processing . they find that language models are able to distinguish verbs that appear in plausible and implausible contexts . |
| Outcome: | The proposed models are able to distinguish verbs that appear in plausible and implausible contexts, but none captures the same selective patterns that human comprehenders exhibit during real-time verb prediction. |
Are Transformers a Modern Version of ELIZA? Observations on French Object Verb Agreement (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that unsupervised sentence representations of neural networks encode syntactic information by observing that neural language models are able to predict the agreement between a verb and its subject. |
| Approach: | They propose to take an alternative look at these results by studying whether neural networks are able to build an abstract sentence representation rather than capture surface statistical regularities. |
| Outcome: | The proposed model can achieve high accuracy on the long-range French object-verb agreement, indicating a possible flaw in the model's syntactic ability. |
Language Models at the Syntax-Semantics Interface: A Case Study of the Long-Distance Binding of Chinese Reflexive Ziji (2025.coling-main)
Copied to clipboard
| Challenge: | Existing language models tend to rely heavily on sequential cues, but not always favoring the closest strings. |
| Approach: | They construct a dataset of 320 synthetic sentences and 360 natural sentences from the BCC corpus . they evaluate 21 language models against this dataset and compare their performance to native Mandarin speakers . |
| Outcome: | The proposed models do not replicate human-like judgments in Mandarin Chinese . the results show that existing models tend to rely heavily on sequential cues . |
Exploring Statistical and Neural Models for Noun Ellipsis Detection and Resolution in English (2020.aacl-srw)
Copied to clipboard
| Challenge: | Existing approaches to noun ellipsis resolution have been sparse, using syntactic feature constraints for marking licensors and selecting their antecedents. |
| Approach: | They propose to use supervised machine learning to improve the existing F1 score by 16.55% and resolution by 14.97% for noun ellipsis subtasks. |
| Outcome: | The proposed framework improves the existing F1 score by 16.55% and the resolution subtask by 14.97%. |
Filling in the Mechanisms: How do LMs Learn Filler-Gap Dependencies under Developmental Constraints? (2026.findings-acl)
Copied to clipboard
| Challenge: | Language models lack language-specific biases, yet still posit some important syntactic generalizations. |
| Approach: | They applied Distributed Alignment Search to checkpoints of a language model from the BabyLM challenge to evaluate whether representations of filler-gap dependencies transfer between wh-questions and topicalization. |
| Outcome: | The results suggest shared, yet item-sensitive mechanisms may develop with limited training data. |
Typological Features for Multilingual Delexicalised Dependency Parsing (N19-1)
Copied to clipboard
| Challenge: | Existing universal models to describe the syntax of languages are debated for decades . a new study examines the plausibility of universal grammars in dependency parsing . |
| Approach: | They propose to use typological features to describe the syntax of languages to train a multilingual dependency parser. |
| Outcome: | The proposed model can be trained on 40 languages with the help of typological features. |
The Curious Case of Control (2022.emnlp-main)
Copied to clipboard
| Challenge: | Normally-developing children struggle with subject control clauses long after they have acquired the components to understand them. |
| Approach: | They examine whether heuristics based on semantic roles are consistent with children's English . they find that models are more sensitive to agent-patient information . |
| Outcome: | The results show that models fail on subject control but fail on object control . the authors show that raising salience of agent and patient relations results in significant changes . |
Ellipsis Resolution as Question Answering: An Evaluation (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing models for ellipsis resolution in English are expensive and cumbersome . ellipas are hard, open problems in NLP, and can cause errors in translation, question answering, and dialogue understanding. |
| Approach: | They propose an alternative approach to ellipsis resolution based on question answering architectures. |
| Outcome: | The proposed model outperforms the current state of the art for ellipsis resolution in English . it shows that annotations can be useful for a subset of the known ellipas . |
Understanding Cross-Lingual Alignment—A Survey (2024.findings-acl)
Copied to clipboard
| Challenge: | Cross-lingual alignment is the meaningful similarity of representations across languages in multilingual language models. |
| Approach: | They propose a taxonomy of methods to improve cross-lingual alignment . they argue that an effective trade-off between language-neutral and language-specific information is key . |
| Outcome: | The proposed methods can be applied to encoder models and encoder-decoder-only models . they show that language-neutral and language-specific information is key . |
How Abstract Is Linguistic Generalization in Large Language Models? Experiments with Argument Structure (2023.tacl-1)
Copied to clipboard
| Challenge: | Competent speakers of a language know how likely a word w is to appear in a specific context . |
| Approach: | They use transformer-based large language models to generalize a novel noun argument . they show a bias to generalise based on linear order, instead of a linear order . |
| Outcome: | The proposed models perform well in generalizing the distribution of a novel noun argument between related contexts that were seen during pre-training. |