Papers by Song Duong
LOCOST: State-Space Models for Long Document Abstractive Summarization (2024.eacl-long)
Copied to clipboard
Florian Le Bronnec, Song Duong, Mathieu Ravaut, Alexandre Allauzen, Nancy Chen, Vincent Guigue, Alberto Lumbreras, Laure Soulier, Patrick Gallinari
| Challenge: | State-space models are a low-complexity alternative to transformers for text generation . however, the quadratic complexity of the input length restricts the application of large pretrained models to long texts. |
| Approach: | They propose an encoder-decoder architecture based on state-space models for conditional text generation with long context inputs. |
| Outcome: | The proposed model saves memory and memory during training and inference time while saving 50% and 87% of memory. |