Papers by David Friede
AgentQuest: A Modular Benchmark Framework to Measure Progress and Improve LLM Agents (2024.naacl-demo)
Copied to clipboard
Luca Gioacchini, Giuseppe Siracusano, Davide Sanvito, Kiril Gashteovski, David Friede, Roberto Bifulco, Carolin Lawrence
| Challenge: | Existing benchmarks are narrow and simply compute overall task success. |
| Approach: | They propose a framework where both benchmarks and metrics are modular and easily extensible through well documented and easy-to-use APIs. |
| Outcome: | The proposed framework can track agent progress on two use cases and identify common failure points and refine the agent architecture to obtain a significant performance increase. |