| Challenge: | Construction grammar posits that constructions are form-meaning pairings that are acquired through experience with language. |
| Approach: | They propose to use a RoBERTa model to model constructions as patterns of statistical affinity . they show that statistical affinity is likely an important, but partial, signal available to learners . |
| Outcome: | The proposed model shows that constructions will be revealed as patterns of statistical affinity . the proposed model is based on a model that is able to distinguish constructions from text . |
Similar Papers
BabyLM’s First Constructions: Causal interventions provide a signal of learning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows sensitivity to constructions in pretrained language models, but their relevance to human language learning is doubted. |
| Approach: | They use construction grammars to demonstrate sensitivity to constructions in pretrained language models. |
| Outcome: | The proposed models learn diverse constructions even hard cases that are superficially indistinguishable. |
A Construction Grammar Corpus of Varying Schematicity: A Dataset for the Evaluation of Abstractions in Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been developed without a theoretical framework . evaluating and improving LLMs will benefit from theoretical frameworks that enable comparison of structures of human language and model of language built up by LLM. |
| Approach: | They propose to use a construction grammar schema corpus to compare human grammar to LLMs' model of language. |
| Outcome: | The proposed corpus shows that even the largest LLMs are limited to more substantive constructions and do not recognize similarity of purely schematic constructions. |
Function Words as Statistical Cues for Language Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies have argued that function words aid learning abstract grammatical knowledge from linear input. |
| Approach: | They examine the statistical distribution of function words and their properties . they show that function words are reliable, diverse, and informative . |
| Outcome: | The results show that function words preserve high frequency, reliable syntactic association, phrase-boundary alignment and are informative to structural dependency. |
What Can String Probability Tell Us About Grammaticality? (2026.tacl-1)
Copied to clipboard
| Challenge: | linguistic theories have argued that language models have largely achieved grammatical competence, but they will assign non-zero probability to all strings. |
| Approach: | They propose a theoretical framework for analyzing string probabilities in linguistics based on simple assumptions about the generative process of corpus data. |
| Outcome: | The proposed framework makes three predictions using 280K sentence pairs in English and Chinese. |
Unpacking Let Alone: Human-Scale Models Generalize to a Rare Construction in Form but not Meaning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent evidence suggests that language models with human-scale pretraining data may possess a similar generalization ability by generalizing from frequent to rare constructions. |
| Approach: | They construct a synthetic benchmark that targets syntactic and semantic properties of the English Let-Alone construction and compare it with a human-scale transformer language model. |
| Outcome: | The proposed model can generalize from frequent to rare constructions, but human-scale models do not make correct generalizations about Let-Alone’s meaning. |
CxGBERT: BERT meets Construction Grammar (2020.coling-main)
Copied to clipboard
| Challenge: | lexico-semantic elements capture a large amount of linguistic information, but they do not capture all information contained in text. |
| Approach: | They propose to use BERT to train a model that uses a deep bidirectional transformer to capture a significant amount of lexico-semantic information. |
| Outcome: | The proposed model captures lexico-semantic information, but it is redundantly encoded in lexical information. |
Why is penguin more similar to polar bear than to sea gull? Analyzing conceptual knowledge in distributional models (2020.acl-srw)
Copied to clipboard
| Challenge: | Several analysis methods have been shown to be limited and are not well understood . thesis aims to understand distributional semantic representations based on linguistic data . |
| Approach: | They propose a framework for investigating the information encoded in distributional semantic models . they combine observations made on corpora with insights obtained from data manipulation experiments . |
| Outcome: | The proposed framework pairs observations made on corpora with insights obtained from data manipulation experiments. |
A Method for Studying Semantic Construal in Grammatical Constructions with Interpretable Contextual Embedding Spaces (2023.acl-long)
Copied to clipboard
| Challenge: | Existing paradigms for the linguistically oriented exploration of large neural language models include treating the model as a linguistic test subject by measuring output on test sentences and building probing classifiers on top of embeddings to test whether the embeddables are sensitive to certain properties like dependency structure. |
| Approach: | They project contextual embeddings into interpretable semantic spaces, each defined by a different set of psycholinguistic feature norms. |
| Outcome: | The proposed method can probe the distributional meaning of syntactic constructions at a templatic level, abstracted away from specific lexemes. |
Using Collostructional Analysis to evaluate BERT’s representation of linguistic constructions (2023.findings-acl)
Copied to clipboard
| Challenge: | Collostructional analysis is a technique devised to find correlations between words and linguistic constructions in order to analyse meaning associations of these constructions. |
| Approach: | They propose to compare English BERT’s meaning representations to known constructions from the linguistics literature by predicting words that can be used in open slots of constructions and finding similar sequences using BERT's output embeddings. |
| Outcome: | The proposed methods show that English BERT represents constructional meaning to a certain extent, but does not separate instances of a construction from a near-synonymous construction that has a different form. |
Building Static Embeddings from Contextual Ones: Is It Useful for Building Distributional Thesauri? (2022.lrec-1)
Copied to clipboard
| Challenge: | contextual language models are dominant in the field of Natural Language Processing, but they are not suitable for all uses. |
| Approach: | They propose a method for building word or type-level embeddings from contextual models . they evaluate a large set of English nouns from the perspective of extracting semantic similarity relations . |
| Outcome: | The proposed method can be used to build word or type embeddings from contextual models . it can be exploited for a wide set of English nouns, showing it can improve distributional thesauri . |