To Split or Not to Split: Composing Compounds in Contextual Vector Spaces (2023.emnlp-main)
Copied to clipboard
| Challenge: | Contextual word embedding models rely on sub-word tokenization to represent single orthographic words but are often suboptimal in under-resourced contexts. |
| Approach: | They propose to use a masked language modelling task to evaluate the model's performance . they use re-trained tokenizers to pre-split compounds into constituents . |
| Outcome: | The proposed models improve on the masked language modelling task and compositionality prediction by pre-splitting compounds into constituents. |
Similar Papers
Modeling the Evolution of English Noun Compounds with Feature-Rich Diachronic Compositionality Prediction (2025.acl-long)
Copied to clipboard
| Challenge: | Empirical research directly addressing these issues is limited to a small number of studies suggesting that compounding is a highly productive process. |
| Approach: | They represent English noun compounds as vectors of time-specific values and implement a set of features to classify them for present-day compositionality and assess the informativeness of the corresponding linguistic patterns. |
| Outcome: | The proposed method captures relevant and complementary information across approaches and shows that low-compositional meanings are reflected by a parallel drop in compositionality and sustained semantic change. |
Variants of Vector Space Reductions for Predicting the Compositionality of English Noun Compounds (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing approaches to predict the degree of compositionality of noun compounds are based on comparing compounds and their constituents within a vector space and using distributional similarity as a proxy to predict their degree of semantic relatedness. |
| Approach: | They propose to use distributional similarity as a proxy to predict the semantic relatedness between the compounds and their constituents as the compound’s degree of compositionality. |
| Outcome: | The proposed methods are most successful and stable in terms of dimensionality and part-of-speech reductions. |
Splintering Nonconcatenative Languages for Better Tokenization (2025.findings-acl)
Copied to clipboard
| Challenge: | Subword tokenizers are not effective in nonconcatenative languages such as Hebrew and Arabic, where morphology is encoded in root-template patterns, or Malay and Georgian, where split affixes are common. |
| Approach: | They propose a pre-processing step which rearranges text into a linear form that better represents nonconcatenative morphologies. |
| Outcome: | The proposed method is based on measures evaluating token vocabularies in Hebrew, Arabic, and Malay, and on BERT-architecture models trained for Hebrew. |
CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Currently, there is no dataset containing compound and non-compound words across languages . however, current LLMs perform poorly on words tokenized unfavorably by subword tokenization. |
| Approach: | They propose to use a Wiktionary dataset to evaluate large language models on decompounding . they find that current LLMs perform poorly on words tokenized unfavorably . |
| Outcome: | The proposed model outperforms the best unsupervised models by 13.9% accuracy on average. |
A Systematic Search for Compound Semantics in Pretrained BERT Architectures (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing models for noun compounds have been less successful in predicting compositionality than transformers . authors: suboptimal use of encoded information may be a contributing factor . performance of transformer-based models is poor, authors say . |
| Approach: | They propose to use semantic knowledge derived from pretrained BERT to predict compositionality . they find distinct linguistic roles of heads and modifiers are reflected by differences in BERT representations . |
| Outcome: | The proposed model improves on unsupervised implementations of pretrained BERT . empirical properties such as frequency, productivity, and ambiguity affect performance . |
A Joint Approach to Compound Splitting and Idiomatic Compound Detection (2020.lrec-1)
Copied to clipboard
| Challenge: | Compounding is a common word-formation process in Germanic languages . high productivity and low corpus frequency of compounds increase vocabulary size . |
| Approach: | They develop a deep learning-based approach to noun compound splitting and idiomatic compound detection for the German language. |
| Outcome: | The proposed approach outperforms the current state of the art in noun compound splitting and idiomatic compound detection for the German language. |
What Can Diachronic Contexts and Topics Tell Us about the Present-Day Compositionality of English Noun Compounds? (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to determine the semantic relatedness between compounds and constituents have applied a synchronic perspective, but this study examines what diachronic changes in contexts and semantic topics reveal about the compounds’ present-day compositionality. |
| Approach: | They propose to use two diachronic vector spaces to model compositional patterns between compounds with low and high present-day compositionality. |
| Outcome: | The proposed model performs on par with co-occurrence space and captures similar information. |
Representing Compounding with OntoLex. An Evaluation of Vocabularies for Word Formation Resources (2024.lrec-main)
Copied to clipboard
| Challenge: | OntoLex is a de facto standard for the modelling of lexical resources in the framework of Linguistic Linked Open Data. |
| Approach: | They propose to use OntoLex to convert Linked Open Data into compounds by using the RDF model. |
| Outcome: | The proposed model can be applied to all resources harmonized in that format, potentially allowing for the conversion into Linked Open Data of a large amount of structured data. |
Can Large Language Models Interpret Noun-Noun Compounds? A Linguistically-Motivated Study on Lexicalized and Novel Compounds (2024.acl-long)
Copied to clipboard
| Challenge: | Noun-noun compounds represent an important challenge for Natural Language Understanding . correct interpretation of noun-nomin compounds is essential for many applications . |
| Approach: | They test whether Large Language Models can interpret the semantic relation between nouns . they also test whether they can abstract from such knowledge to predict the relation . |
| Outcome: | The proposed models can interpret the semantic relation between nouns and compounds using analogical comparisons. |
Improving Tokenisation by Alternative Treatment of Spaces (2022.emnlp-main)
Copied to clipboard
| Challenge: | Subword tokenisation is a key initial step in processing natural language . it uses a number of different methods to tokenise text, including a stringsearching technique and a word-matching technique. |
| Approach: | They propose to use a vocabulary-based approach to tokenise text using a numerical ID and a mathematical function to manipulate it. |
| Outcome: | The method is based on a set of training data and learning from it to build a vocabulary and tokenise it at inference time using this vocabulary and learnt parameters. |