Learning Efficient Task-Specific Meta-Embeddings with Word Prisms (2020.coling-main)
Copied to clipboard
| Challenge: | Word embeddings possess different lexical properties depending on the notion of context defined at training time. |
| Approach: | They introduce a meta-embedding method that learns to combine source embeddings according to the task at hand. |
| Outcome: | The proposed method improves performance on six extrinsic evaluations over other methods. |
Similar Papers
Learning Word Meta-Embeddings by Autoencoding (C18-1)
Copied to clipboard
| Challenge: | Existing word embeddings have shown superior performance in numerous Natural Language Processing (NLP) tasks, however, their performances vary significantly across different tasks. |
| Approach: | They propose to combine distributed word embeddings to produce more accurate and complete meta-embeddings of words. |
| Outcome: | The proposed meta-embeddings outperform the state-of-the-art in multiple tasks. |
Dynamic Meta-Embeddings for Improved Sentence Representations (D18-1)
Copied to clipboard
| Challenge: | A sprawling literature has emerged about what word embeddings are most useful for which tasks . word embed-ding is a technique that can be used to learn word-level meaning representations for a variety of tasks. |
| Approach: | They propose a method for supervised learning of embedding ensembles that leads to state-of-the-art performance on a variety of tasks. |
| Outcome: | The proposed method leads to state-of-the-art performance on a variety of tasks. |
Task-oriented Domain-specific Meta-Embedding for Text Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods neglect domain-specific knowledge and use the same word embedding for each word in all domain-specified datasets. |
| Approach: | They propose a method to incorporate domain-specific and task-oriented information into meta-embeddings by combining pre-trained word embeddings. |
| Outcome: | The proposed method performs well on four text classification datasets and shows that it is compatible with existing methods. |
Frustratingly Easy Meta-Embedding – Computing Meta-Embeddings by Averaging Source Word Embeddings (N18-2)
Copied to clipboard
| Challenge: | Existing methods for producing word embeddings have shown to produce accurate meta-embeddings from pre-trained source embeddables. |
| Approach: | They propose to use arithmetic mean of two distinct word embedding sets to produce an accurate meta-embedding. |
| Outcome: | The proposed method produces meta-embeddings comparable or better than more complex methods. |
Are Word Embeddings Really a Bad Fit for the Estimation of Thematic Fit? (2020.lrec-1)
Copied to clipboard
| Challenge: | In recent years, vectors derived from neural network training have replaced count-based distributional semantic models as a de facto standard for word representation in NLP. |
| Approach: | They propose to evaluate count models and word embeddings on thematic fit estimation by taking into account a larger number of parameters and verb roles and introducing dependency-based embedders in the comparison. |
| Outcome: | The proposed model outperforms count models and word embeddings in thematic fit estimation tasks while introducing dependency-based embedders. |
Leveraging Meta-Embeddings for Bilingual Lexicon Extraction from Specialized Comparable Corpora (C18-1)
Copied to clipboard
| Challenge: | Recent studies on bilingual lexicon extraction from specialized comparable corpora show differences in performance . lack of large specialized corporan to build efficient representations can be partially explained . |
| Approach: | They propose to use character-based embedding models to combine different embeddable models . they emphasize how character-driven embeddance models outperform other models on quality . |
| Outcome: | The proposed model outperforms other models on quality of extracted bilingual lexicons . comparable corpora are an interesting and practical alternative to parallel corporation . |
A Deeper Look into Dependency-Based Word Embeddings (N18-4)
Copied to clipboard
| Challenge: | Word embeddings trained with dependency contexts excel at different tasks, and enhanced dependencies often improve performance. |
| Approach: | They propose to use dependency-based word embeddings to capture semantic similarity rather than relatedness. |
| Outcome: | The results show that word embeddings trained with Universal and Stanford dependencies excel at different tasks and that enhanced dependencies often improve performance. |
Sentence Meta-Embeddings for Unsupervised Semantic Textual Similarity (2020.acl-main)
Copied to clipboard
| Challenge: | Existing word embeddings combine complementary strengths of their components to achieve unsupervised semantic similarity (STS). |
| Approach: | They propose to ensemble pre-trained sentence encoders into sentence meta-embeddings to achieve unsupervised Semantic Textual Similarity (STS) they adapt dimensionality reduction, generalized Canonical Correlation Analysis and cross-view auto-encoders to their work. |
| Outcome: | The proposed method achieves 3.7% to 6.4% Pearson’s r over single-source word embeddings on the STS Benchmark and on the StS12-STS16 datasets. |
Are the Best Multilingual Document Embeddings simply Based on Sentence Embeddings? (2023.findings-eacl)
Copied to clipboard
| Challenge: | obtaining document embeddings at document level is challenging due to computational requirements and lack of appropriate data. |
| Approach: | They compare methods to produce document-level representations from sentences based on LASER, LaBSE, and Sentence BERT pre-trained multilingual models. |
| Outcome: | The proposed methods produce document-level representations from sentences in 8 languages . the results show that a clever combination of sentence embeddings is usually better than encoding the full document as a single unit. |
Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms (P18-1)
Copied to clipboard
Dinghan Shen, Guoyin Wang, Wenlin Wang, Martin Renqiang Min, Qinliang Su, Yizhe Zhang, Chunyuan Li, Ricardo Henao, Lawrence Carin
| Challenge: | Existing deep learning architectures to model compositionality in text sequences require a large number of parameters and expensive computations. |
| Approach: | They propose two additional pooling strategies over word embeddings for improved interpretability and hierarchical pooling for spatial (n-gram) information within text sequences. |
| Outcome: | The proposed pooling strategies improve interpretability and preserve spatial (n-gram) information within text sequences. |