Papers by Tinne Tuytelaars
Is this chart lying to me? Automating the detection of misleading visualizations (2026.acl-long)
Copied to clipboard
| Challenge: | Prior work has shown that both humans and MLLMs are frequently deceived by misleading visualizations. |
| Approach: | They propose a benchmark of 2,604 real-world visualizations annotated with 12 types of misleaders. |
| Outcome: | The proposed framework can detect misleading visualizations and identify specific design rules they violate . the proposed framework is based on a synthetic dataset of 81,814 visualizations . |
Learning to Route for Dynamic Adapter Composition in Continual Learning with Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work shows that PEFT methods can be competitive with, or even superior to, full fine-tuning of PLMs. |
| Approach: | They propose a method that isolates the training of new PEFT modules to ensure their task specialization and learns to compose them by training a network of routers that leverages a small memory containing examples of previously seen tasks. |
| Outcome: | The proposed method improves generalization and performance in two CL setups. |
Visually-Aware Context Modeling for News Image Captioning (2024.naacl-long)
Copied to clipboard
| Challenge: | a new framework for News Image Captioning emphasizes the connection between textual context and visual elements. |
| Approach: | They propose a face-naming module for learning better name embeddings from news images . they use CLIP to retrieve sentences that are semantically close to the image . |
| Outcome: | The proposed framework outperforms the current state-of-the-art by 7.97/5.80 CIDEr scores on GoodNews/NYTimes800k. |
OASIS: Online Sample Selection for Continual Instruction Tuning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for continual instruction tuning (CIT) use pre-trained reference models, which are impractical in CIT setups since future data are unknown. |
| Approach: | They propose an adaptive online sample selection approach that estimates each sample’s informativeness relative to all previously seen data and minimizes informative redundancy through iterative selection score updates. |
| Outcome: | Experiments on various large foundation models show that using only 25% of the data achieves comparable performance to full-data training and outperforms the state-of-the-art sampling methods. |
Navigating the Nuances: A Fine-grained Evaluation of Vision-Language Navigation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a new evaluation framework for vision-language navigation is proposed . current evaluation standards hinge on endpoint success rates and path alignment metrics . |
| Approach: | They propose a semi-automatic method for CFG construction with Large-Language Models . they induct data spanning five principal instruction categories and analyze them . |
| Outcome: | The proposed framework diagnoses current models for the Vision-Language Navigation task at a finer-grained level. |
Decoding Language Spatial Relations to 2D Spatial Arrangements (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Using a model architecture, we decode text to 2D spatial arrangements in a multi-object and multi-relationship setting. |
| Approach: | They propose a model architecture Spatial-Reasoning Bert that decodes language to 2D spatial arrangements in a multi-object and multi-relationship setting. |
| Outcome: | The proposed model can generate complete abstract scenes if paired with a clip-arts predictor and can generalize to out-of-sample data to a reasonable extent. |
Protecting multimodal large language models against misleading visualizations (2026.acl-long)
Copied to clipboard
| Challenge: | MLLMs are robust to misleading visualizations, i.e., charts that distort the underlying data, leading readers to draw inaccurate conclusions. |
| Approach: | They propose to use table-based QA and redrawing the visualization to improve QA performance on misleading visualizations. |
| Outcome: | The proposed methods improve MLLM question-answering accuracy on misleading visualizations without compromising accuracy on non-misleading ones. |