Papers by Matej Martinc
Leveraging Contextual Embeddings for Detecting Diachronic Semantic Shift (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for word embeddings have been used to model semantic relations with word embeds. |
| Approach: | They propose a method that leverages contextual embeddings for diachronic semantic shift detection by generating time specific word representations from BERT embedds. |
| Outcome: | The proposed method performs comparable to the current state-of-the-art without time consuming domain adaptation on large corpora. |
Embeddings models for Buddhist Sanskrit (2022.lrec-1)
Copied to clipboard
| Challenge: | Despite extensive scholarly endeavors, much uncertainty still surrounds this body of literature, especially regarding matters of chronology, authorship, compositional history. |
| Approach: | They propose a corpus of Buddhist texts, a general corpus and word similarity and word analogy datasets for embeddings models. |
| Outcome: | The proposed models perform better on semantic similarity and word analogy tasks than on contextual models. |
Reusable workflows for gender prediction (L18-1)
Copied to clipboard
| Challenge: | Existing systems for author profiling (AP) modeling require extensive feature engineering and testing. |
| Approach: | They propose to implement a system for author profiling (AP) modeling that reduces the complexity and time of building a sophisticated model for a number of different AP tasks. |
| Outcome: | The proposed model achieves comparable results to state of the art models for cross-genre gender prediction, but lags when genre of test set is different from genre of train set. |
SEKE: Specialised Experts for Keyword Extraction (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Keyword extraction involves identifying the most descriptive words in a document . supervised keyword extraction is based on the mixture of experts (MoE) technique . |
| Approach: | They propose a supervised keyword extraction approach based on the mixture of experts technique . they use a learnable routing sub-network to direct information to specialised experts . |
| Outcome: | The proposed approach is based on the mixture of experts (MoE) technique . experts attend to each token and integrate it with a bidirectional long-term memory network . |
Out of Thin Air: Is Zero-Shot Cross-Lingual Keyword Detection Better Than Unsupervised? (2022.lrec-1)
Copied to clipboard
| Challenge: | Keyword extraction is the task of retrieving words that are essential to the content of a document. |
| Approach: | They propose to use pretrained multilingual language models for zero-shot cross-lingual keyword extraction on low-resource languages with limited or no available labeled training data. |
| Outcome: | The proposed models outperform state-of-the-art unsupervised methods on low-resource languages with limited or no training data. |
Scalable and Interpretable Semantic Change Detection (2021.naacl-main)
Copied to clipboard
| Challenge: | Several cluster-based methods for word usage change detection are unscalable in terms of memory consumption and computation time. |
| Approach: | They propose a scalable method for word usage-change detection that uses contextual embeddings to aggregate word usages into clusters. |
| Outcome: | The proposed method offers high performance and interpretability while being unscalable. |
Effectiveness of Data Augmentation and Pretraining for Improving Neural Headline Generation in Low-Resource Settings (2022.lrec-1)
Copied to clipboard
| Challenge: | Neural approaches for natural language generation (NLG) have mushroomed due to large textual resources. |
| Approach: | They propose to use a pretrained multilingual encoder-decoder model and a combination of two pretrained language models to train a model in a low-resource setting. |
| Outcome: | The proposed model outperforms the previous model on English and on a small subset of the same data. |