Papers by Nicolas Chapados
XC-Cache: Cross-Attending to Cached Context for Efficient LLM Inference (2024.findings-emnlp)
Copied to clipboard
Joao Monteiro, Étienne Marcotte, Pierre-Andre Noel, Valentina Zantedeschi, David Vazquez, Nicolas Chapados, Christopher Pal, Perouz Taslakian
| Challenge: | XC-Llama uses pre-trained decoder-only models to condition generation on reference text without the prompt. |
| Approach: | They propose a model that uses cross-attention to condition generation on reference text without the prompt. |
| Outcome: | The proposed models outperform prompt-based inference methods and reduce space footprint relative to standard KV caching by two orders of magnitude. |