Papers by Gautier Viaud
GroUSE: A Benchmark to Evaluate Evaluators in Grounded Question Answering (2025.coling-main)
Copied to clipboard
| Challenge: | Existing automated RAG evaluation frameworks overlook important failure modes when using GPT-4 as a judge. |
| Approach: | They propose a novel pipeline to assess the calibration and discrimination capabilities of judge models by using a meta-evaluation benchmark of 144 unit tests to identify key failure modes. |
| Outcome: | The proposed pipeline improves on existing frameworks, while state-of-the-art open-source judges do not generalize to their proposed criteria. |
Context is Gold to find the Gold Passage: Evaluating and Training Contextual Document Embeddings (2025.emnlp-main)
Copied to clipboard
| Challenge: | Modern document retrieval embedding methods typically encode passages (chunks) from documents independently, often overlooking contextual information from the rest of the document. |
| Approach: | They propose a benchmark to evaluate retrieval models' ability to leverage document-wide context. |
| Outcome: | The proposed method significantly improves retrieval quality on ConTEB without sacrificing base model performance. |
FQuAD2.0: French Question Answering and Learning When You Don’t Know (2022.lrec-1)
Copied to clipboard
| Challenge: | Question Answering, including Reading Comprehension, has seen significant scientific breakthroughs over the past few years . but most of these breakthroughs are centered on the English language . |
| Approach: | They propose a dataset to train Question Answering models in the French language . they extend the dataset to 17,000+ unanswerable questions annotated adversarially . |
| Outcome: | The proposed dataset makes it possible to train French Question Answering models with the ability to distinguish unanswerable questions from answerable ones. |
ViDoRe V3: A Comprehensive Evaluation of Retrieval Augmented Generation in Complex Real-World Scenarios (2026.acl-long)
Copied to clipboard
António Loison, Quentin Macé, Antoine Edy, Victor Xing, Tom Balough, Gabriel de Souza P. Moreira, Bo Liu, Manuel Faysse, Celine Hudelot, Gautier Viaud
| Challenge: | Existing benchmarks focus on textual data, single-document comprehension, or evaluating retrieval and generation in isolation. |
| Approach: | They propose a multimodal RAG benchmark featuring multi-type queries over visually rich document corpora. |
| Outcome: | The proposed benchmark outperforms existing benchmarks in visual retrieval and human-verified queries. |
Revisiting Instruction Fine-tuned Model Evaluation to Guide Industrial Applications (2023.emnlp-main)
Copied to clipboard
| Challenge: | Instruction fine-tuned (IFT) models are gaining traction in industrial NLP to unlock task-specific performance gains and strengthen model alignment with industry requirements. |
| Approach: | They propose to use instruction fine-tuned (IFT) models to enhance the zero-shot capabilities of Large Language Models (LLMs) they also propose to leverage IFT models to analyze the trade-offs that emerge in industrial settings. |
| Outcome: | The proposed model is well adapted to new evaluation metric requirements, and offers practical insights for real-world LLM deployment. |