Papers by Camille Couturier
Hybrid-RACA: Hybrid Retrieval-Augmented Composition Assistance for Real-time Text Prediction (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Large language models (LLMs) enhanced with retrieval augmentation have shown great performance in many applications, but their computational overhead and additional retrieval step limit their effectiveness in real-time tasks. |
| Approach: | They propose a system that combines a cloud-based LLM with a smaller client-side model through retrieval augmented memory to provide real-time text prediction. |
| Outcome: | The proposed system can generate better responses from the cloud-based model while maintaining low latency. |