Papers by Eric Darve
Universal Sentence Representation Learning with Conditional Masked Language Model (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to learn sentence representations on unlabeled corpora are difficult and expensive to obtain, making it hard to cover many domains and languages. |
| Approach: | They propose a method to train sentence representations on large unlabeled corpora by conditioning on the encoded vectors of adjacent sentences. |
| Outcome: | The proposed method outperforms existing models on SentEval and can be extended to a broad range of languages and domains. |
A Simple and Effective Method To Eliminate the Self Language Bias in Multilingual Representations (2021.emnlp-main)
Copied to clipboard
| Challenge: | Language agnostic and semantic-language information isolation is an emerging research direction for multilingual representations models. |
| Approach: | They propose a method that factors out language identity information from semantic related components in multilingual representations pre-trained on monolingual data. |
| Outcome: | The proposed method improves cross-lingual transfer performance on weak alignment models. |
Embedding Imputation with Grounded Language Information (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to embedding imputation use vector space properties or subword information to learn representations for rare or unseen words. |
| Approach: | They propose an online method to construct a knowledge graph from grounded information and an algorithm to map from the resulting graph to the space of the pre-trained embeddings. |
| Outcome: | The proposed method improves on a card-660 task by 11% and 17.8% respectively using GloVe embeddings. |
TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing abstractive summarization models ignore abundant unlabeled corpora resources . TED outperforms all unsupervised abstractive baselines on NYT, CNN/DM and English Gigaword datasets . |
| Approach: | They propose a transformer-based unsupervised text summarization system with pretraining on large-scale data. |
| Outcome: | The proposed system outperforms baseline models on NYT, CNN/DM and English Gigaword datasets with various document styles. |