Papers by Christoph Leiter
EffEval: A Comprehensive Evaluation of Efficiency for MT Evaluation Metrics (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a recent surge of interest in developing evaluation metrics based on pretrained large language models (LLMs) can better cope with lexical variation. |
| Approach: | They propose to replace computation-intensive transformers with lighter alternatives and employ linear and quadratic approximations for alignment algorithms on top of LLM representations. |
| Outcome: | The proposed approach replaces computation-intensive transformers with lighter alternatives and employs linear and quadratic approximations for alignment algorithms on top of LLM representations. |
BMX: Boosting Natural Language Generation Metrics with Explainability (2024.findings-eacl)
Copied to clipboard
| Challenge: | Modern language model (LM) based natural language generation evaluation metrics achieve astonishing results in grading machine generated sentences like humans would. |
| Approach: | They use word-level explanations to convert a word- level score into a segment-level score and combine this segment- level with the original metric to obtain a better metric. |
| Outcome: | The proposed metrics improve across MT and summarization datasets while achieving small improvements on machine translation. |
GerAV: Towards New Heights in German Authorship Verification using Fine-Tuned LLMs on a New Benchmark (2026.findings-acl)
Copied to clipboard
| Challenge: | Authorship verification (AV) is a task of determining whether two texts were written by the same author. |
| Approach: | They propose a benchmark for German AV comprising over 400k labeled text pairs. |
| Outcome: | The proposed model outperforms baselines and state-of-the-art models by 0.09 and surpasses GPT-5 in a zero-shot setting by 0.08. |
PrExMe! Large Scale Prompt Exploration of Open Source LLMs for Machine Translation and Summarization Evaluation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are useful for low-resource scenarios and time-restricted applications. |
| Approach: | They propose a large-scale evaluation tool for large language models that uses prompts . they evaluate 720 prompt templates for open-source LLM-based metrics on MT and summarization datasets a 6.6M evaluations. |
| Outcome: | The proposed model evaluates 720 prompt templates on machine translation and summarization datasets. |