Papers by Ivan Rep
Are ELECTRA’s Sentence Embeddings Beyond Repair? The Case of Semantic Textual Similarity (2024.findings-emnlp)
Copied to clipboard
| Challenge: | ELECTRA's sentence embeddings are poorer than BERT's, resulting in a significant drop in performance for semantic textual similarity (STS). |
| Approach: | They propose to use a truncated model fine-tuning method to repair the embeddings by reducing the number of parameters and producing smaller embeddables. |
| Outcome: | The proposed method improves Spearman correlation coefficient by over 8 points while increasing parameter efficiency on the STS Benchmark. |