Papers by Quentin Fournier
Exploring Quantization for Efficient Pre-Training of Transformer Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Quantization has proven to be effective after pre-training and during fine-tuning, but its effects on pre-trainer performance have remained unexplored. |
| Approach: | They propose a linear quantization strategy to be applied during the pre-training of Transformers to improve model efficiency and stability. |
| Outcome: | The proposed method improves model efficiency, stability, and performance while maintaining language modeling ability. |
Combining Domain and Alignment Vectors Provides Better Knowledge-Safety Trade-offs in LLMs (2025.acl-short)
Copied to clipboard
| Challenge: | Large language models (LLMs) excel in specific technical fields, but are not explicitly trained to be safe. |
| Approach: | They propose a model merging-based alignment method that allows for safer domain-specific models that preserve their utility. |
| Outcome: | The proposed method improves safety alignment on LLMs with minimal degradation on domain-specific benchmarks. |
A Deep Dive into the Trade-Offs of Parameter-Efficient Preference Alignment Techniques (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models are pre-trained on trillions of tokens and instruction-tuned or aligned to specific preferences. |
| Approach: | They propose guidelines to help researchers perform more effective parameter-efficient LLM alignment. |
| Outcome: | The proposed methods outperform preference optimization and outperformed pre-trained models on three key axes. |
Small Encoders Can Rival Large Decoders in Detecting Groundedness (2025.findings-acl)
Copied to clipboard
Istabrak Abbes, Gabriele Prato, Quentin Fournier, Fernando Rodriguez, Alaa Boukhary, Adam Elwood, Sarath Chandar
| Challenge: | Large language models struggle to answer queries reliably when the provided context lacks information, often resorting to ungrounded speculation or internal knowledge. |
| Approach: | They propose to detect whether a given query is grounded in a document provided in context before LLMs generate answers. |
| Outcome: | The proposed model can generate answers that are grounded in the document provided in context while reducing inference latency by orders of magnitude. |