Papers by Ayomide Odumakinde
Multilingual Arbitration: Optimizing Data Pools to Accelerate Multilingual Progress (2025.acl-long)
Copied to clipboard
| Challenge: | Synthetic data generation relies on a single oracle teacher model, which can lead to model collapse and bias propagation. |
| Approach: | They propose a multilingual arbitration approach that exploits performance variations among multiple models for each language. |
| Outcome: | The proposed approach surpasses single-teacher distillation with 80% win rates over proprietary and open-weight models with the largest improvements in low-resource languages. |
Lost in Simulation: LLM-Simulated Users are Unreliable Proxies for Human Users in Agentic Evaluations (2026.acl-long)
Copied to clipboard
| Challenge: | Agentic benchmarks rely on LLM-simulated users to evaluate agent performance . however, the robustness, validity, and fairness of this approach remain unexamined . |
| Approach: | They investigate whether LLM-simulated users are reliable proxies for real human users . they find that agent success rates vary up to 9 percentage points across different LLMs . |
| Outcome: | The results show that simulated users underestimate success on challenging tasks while miscalibrate performance on moderately difficult tasks. |