Papers by Mathieu Ravaut
On Context Utilization in Summarization with Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models excel in abstractive summarization tasks, delivering fluent and pertinent summaries. |
| Approach: | They conduct the first comprehensive study on context utilization and position bias in summarization. |
| Outcome: | The proposed benchmark compares two methods to alleviate position bias in summarization tasks. |
LOCOST: State-Space Models for Long Document Abstractive Summarization (2024.eacl-long)
Copied to clipboard
Florian Le Bronnec, Song Duong, Mathieu Ravaut, Alexandre Allauzen, Nancy Chen, Vincent Guigue, Alberto Lumbreras, Laure Soulier, Patrick Gallinari
| Challenge: | State-space models are a low-complexity alternative to transformers for text generation . however, the quadratic complexity of the input length restricts the application of large pretrained models to long texts. |
| Approach: | They propose an encoder-decoder architecture based on state-space models for conditional text generation with long context inputs. |
| Outcome: | The proposed model saves memory and memory during training and inference time while saving 50% and 87% of memory. |
Unsupervised Summarization Re-ranking (2023.findings-acl)
Copied to clipboard
| Challenge: | Abstractive summarization models have been gaining popularity, but performance of unsupervised models still lags behind supervised models. |
| Approach: | They propose to re-rank summary candidates in an unsupervised manner to close the performance gap between unsupervised and supervised models. |
| Outcome: | The proposed model improves unsupervised models by up to 7.27% and ChatGPT by up 6.86% relative mean ROUGE across four widely-adopted summarization benchmarks. |
SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarization (2022.acl-long)
Copied to clipboard
| Challenge: | Sequence-to-sequence neural networks have enabled great progress in abstractive summarization. |
| Approach: | They propose to train a second-stage model performing re-ranking on a set of summary candidates by using a mixture of experts. |
| Outcome: | The proposed model outperforms the base model on CNN- DailyMail, XSum and Reddit TIFU with a base PEGASUS. |
Towards Summary Candidates Fusion (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for abstractive summarization are limited by the quality of the first-stage candidates. |
| Approach: | They propose a method that fuses several summary candidates to produce a novel abstractive second-stage summary. |
| Outcome: | The proposed method improves ROUGE scores and qualitative properties of fused summaries on several summarization datasets. |
Parameter-Efficient Conversational Recommender System as a Language Processing Task (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing methods to recommend items are categorized into attribute-based and generation-based methods. |
| Approach: | They propose to represent items in natural language and formulate a conversational recommender system that can be optimized in a single stage without relying on non-textual metadata. |
| Outcome: | The proposed model can be optimized in a single stage, without relying on non-textual metadata such as a knowledge graph. |