Papers by Jan Šnajder
Lexical Substitution for Evaluating Compositional Distributional Models (N18-2)
Copied to clipboard
| Challenge: | Compositional Distributional Semantic Models (CDSMs) model the meaning of phrases and sentences in vector space. |
| Approach: | They propose to use lexical substitution to evaluate CDSMs by comparing a LexSub-annotated corpus with a manual LexSub annotation. |
| Outcome: | The proposed model outperforms simple component-wise CDSMs and performs on par with the context2vec LexSub model using the same context. |
Are ELECTRA’s Sentence Embeddings Beyond Repair? The Case of Semantic Textual Similarity (2024.findings-emnlp)
Copied to clipboard
| Challenge: | ELECTRA's sentence embeddings are poorer than BERT's, resulting in a significant drop in performance for semantic textual similarity (STS). |
| Approach: | They propose to use a truncated model fine-tuning method to repair the embeddings by reducing the number of parameters and producing smaller embeddables. |
| Outcome: | The proposed method improves Spearman correlation coefficient by over 8 points while increasing parameter efficiency on the STS Benchmark. |
Sequence Repetition Enhances Token Embeddings and Improves Sequence Labeling with Decoder-only Language Models (2026.findings-eacl)
Copied to clipboard
| Challenge: | Modern language models (LMs) are trained in autoregressive manner, conditioned on the prefix. sequence labeling (SL) tasks assign labels to each individual input token, naturally benefiting from bidirectional context. |
| Approach: | They explore sequence repetition (SR) as a less invasive alternative to decoder-only models . they show that increasing the number of repetitions does not degrade SL performance . |
| Outcome: | The proposed technique improves the quality of token-level embeddings and surpasses encoders and unmasked decoders. |