Papers by Filip Miletic
A Systematic Search for Compound Semantics in Pretrained BERT Architectures (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing models for noun compounds have been less successful in predicting compositionality than transformers . authors: suboptimal use of encoded information may be a contributing factor . performance of transformer-based models is poor, authors say . |
| Approach: | They propose to use semantic knowledge derived from pretrained BERT to predict compositionality . they find distinct linguistic roles of heads and modifiers are reflected by differences in BERT representations . |
| Outcome: | The proposed model improves on unsupervised implementations of pretrained BERT . empirical properties such as frequency, productivity, and ambiguity affect performance . |
Detecting Contact-Induced Semantic Shifts: What Can Embedding-Based Methods Do in Practice? (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on semantic change detection methods has focused on generic research questions and datasets, using them as a training ground for proof-of-concept studies. |
| Approach: | They propose to use type-level embeddings to detect new semantic shifts and token-level embeddeds to isolate regionally specific occurrences. |
| Outcome: | The proposed method is comparable to state-of-the-art on diachrony tasks, but it does not translate to practical value in detecting new semantic shifts. |
To Split or Not to Split: Composing Compounds in Contextual Vector Spaces (2023.emnlp-main)
Copied to clipboard
| Challenge: | Contextual word embedding models rely on sub-word tokenization to represent single orthographic words but are often suboptimal in under-resourced contexts. |
| Approach: | They propose to use a masked language modelling task to evaluate the model's performance . they use re-trained tokenizers to pre-split compounds into constituents . |
| Outcome: | The proposed models improve on the masked language modelling task and compositionality prediction by pre-splitting compounds into constituents. |
Collecting Tweets to Investigate Regional Variation in Canadian English (2020.lrec-1)
Copied to clipboard
| Challenge: | 78.8 million tweets, 1.3 billion words, and a focus on the dialect regions of Toronto, Montreal, and Vancouver are included in this study. |
| Approach: | They propose to use a 78.8-million-tweet, 1.3-billion-word corpus to study regional variation in Canadian English with a focus on the dialect regions of Toronto, Montreal, and Vancouver. |
| Outcome: | The proposed corpus mirrors national and regional specificities of Canadian English and provides sufficient aggregate and user-level data and maintains a reasonably balanced distribution of content across regions and users. |