Papers by Kenan Tang
Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | a high prompt sensitivity has been widely accepted as a core limitation of large language models . a recent study suggests that prompt senescence may be an artifact of evaluation processes . |
| Approach: | They examine whether prompt sensitivity is an inherent weakness or an artifact of evaluation . they find that heuristic evaluation methods overlook semantically correct responses . large language models have achieved remarkable success across a wide range of tasks . |
| Outcome: | The proposed model is more robust to prompt templates than previously thought . the authors show that prompt sensitivity may be an artifact of evaluation rather than a flaw . |
Creative and Context-Aware Translation of East Asian Idioms with GPT-4 (2024.findings-emnlp)
Copied to clipboard
| Challenge: | figurative language is a challenge for human translators, who often choose a context-aware translation . a set of commonly used idioms condenses its figurativ meaning into a few characters . |
| Approach: | They evaluate whether GPT-4 can generate high-quality translations using Pareto-optimal prompting strategies that outperform translation engines from Google and DeepL. |
| Outcome: | The proposed translations outperform translation engines from Google and DeepL at low cost. |