Papers by Joachim Koehler
Judging Quality Across Languages: A Multilingual Approach to Pretraining Data Filtering with Language Models (2025.emnlp-main)
Copied to clipboard
Mehdi Ali, Manuel Brack, Max Lübbering, Elias Wendt, Abbas Goher Khan, Richard Rutmann, Alex Jude, Maurice Kraus, Alexander Arno Weber, Felix Stollenwerk, David Kaczér, Florian Mai, Lucie Flek, Rafet Sifa, Nicolas Flores-Herr, Joachim Koehler, Patrick Schramowski, Michael Fromm, Kristian Kersting
| Challenge: | Existing open-source multilingual datasets rely on heuristic filtering methods restricting both their cross-lingual transferability and scalability. |
| Approach: | They propose a systematic approach that curates diverse and high-quality multilingual data at scale while significantly reducing computational demands. |
| Outcome: | Evaluated empirically across 35 languages, the proposed approach outperforms current heuristic filtering methods like Fineweb2 and improves model training quality and retention rates. |
LLM Parameters for Math Across Languages: Shared or Separate? (2026.acl-srw)
Copied to clipboard
Behzad Shomali, Luisa Victor, Tim Selbach, Ali Hamza Bashir, David Berghaus, Joachim Koehler, Mehdi Ali, Markus Frey
| Challenge: | Existing research on large language models (LLMs) has focused on performance or representational properties, but it remains unclear whether these differences reflect language-specific parameters or a shared mechanism. |
| Approach: | They propose to localize and compare model parameters that support mathematical reasoning across languages. |
| Outcome: | The proposed analysis shows that the model parameters in English and lower-resource languages exhibit partial cross-lingual overlap with systematic language-dependent differences. |