Papers by Vincent Guigue
Separating Retention from Extraction in the Evaluation of End-to-end Relation Extraction (2021.emnlp-main)
Copied to clipboard
| Challenge: | State-of-the-art NLP models adopt shallow heuristics that limit their generalization capability. |
| Approach: | They propose to use heuristics that limit their generalization capability to model lexical overlap with the training set in Named-Entity Recognition and Event or Type heuristic in Relation Extraction to test their models. |
| Outcome: | The proposed model can perform better on the two key tasks, while the retention of training relation triples. |
LOCOST: State-Space Models for Long Document Abstractive Summarization (2024.eacl-long)
Copied to clipboard
Florian Le Bronnec, Song Duong, Mathieu Ravaut, Alexandre Allauzen, Nancy Chen, Vincent Guigue, Alberto Lumbreras, Laure Soulier, Patrick Gallinari
| Challenge: | State-space models are a low-complexity alternative to transformers for text generation . however, the quadratic complexity of the input length restricts the application of large pretrained models to long texts. |
| Approach: | They propose an encoder-decoder architecture based on state-space models for conditional text generation with long context inputs. |
| Outcome: | The proposed model saves memory and memory during training and inference time while saving 50% and 87% of memory. |
Unsupervised Information Extraction: Regularizing Discriminative Approaches with Relation Distribution Losses (P19-1)
Copied to clipboard
| Challenge: | Existing unsupervised relation extraction models are either generative or discriminative . however, they are hard to train without supervision and are unstable . |
| Approach: | They propose a skewness loss and distribution distance loss to improve the performance of discriminative based models. |
| Outcome: | The proposed models surpass current state-of-the-art on three different datasets. |
Let’s Stop Incorrect Comparisons in End-to-end Relation Extraction! (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing literature on Relation Extraction (RE) uses multiple evaluation setups to compare performance. |
| Approach: | They propose to quantify the most common comparison mistake and evaluate it leads to overestimating the final RE performance by around 5% on ACE05. |
| Outcome: | The proposed meta-analysis overestimates the final RE performance by around 5% on ACE05. |
Improving generalization in large langue model by learning prefix subspaces (2023.findings-emnlp)
Copied to clipboard
| Challenge: | emergence of large language models has significantly transformed the applications of deep learning methods in natural language processing. |
| Approach: | They propose to improve LLMs' generalization by optimizing entire models in parameter space by learning entire simplexes of continous prefixes. |
| Outcome: | The proposed method improves generalization of large language models in the scarce data regime. |