Papers by Aonan Zhang
MMAU: A Holistic Benchmark of Agent Capabilities Across Diverse Domains (2025.findings-naacl)
Copied to clipboard
Guoli Yin, Haoping Bai, Shuang Ma, Feng Nan, Yanchao Sun, Zhaoyang Xu, Shen Ma, Jiarui Lu, Xiang Kong, Aonan Zhang, Dian Ang Yap, Yizhe Zhang, Karsten Ahnert, Vik Kamath, Mathias Berglund, Dominic Walsh, Tobias Gindele, Juergen Wiest, Zhengfeng Lai, Xiaoming Simon Wang, Jiulong Shan, Meng Cao, Ruoming Pang, Zirui Wang
| Challenge: | Existing benchmarks focus on specific application scenarios, emphasizing task completion but failing to dissect the underlying skills that drive these outcomes. |
| Approach: | They propose a Massive Multitask Agent Understanding benchmark that evaluates LLMs across five domains and offline tasks. |
| Outcome: | The Massive Multitask Agent Understanding (MMAU) benchmark evaluates models across five domains including Tool-use, Directed Acyclic Graph (DAG) QA, Data Science and Machine Learning coding, Contest-level programming and Mathematics. |
Divide-or-Conquer? Which Part Should You Distill Your LLM? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that Large Language Models (LLMs) can solve reasoning tasks better when they are encouraged to solve subtasks of the main task first. |
| Approach: | They propose a strategy that breaks down reasoning tasks into a problem decomposition phase and a solution phase and propose 'smaller' models that can achieve good generalization. |
| Outcome: | The proposed approach outperforms a single stage solution in two tasks and their impact on reasoning outcomes and inference cost. |