Papers by Daman Arora
Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | The performance of large language models (LLMs) on existing reasoning benchmarks has significantly improved over the past decade. |
| Approach: | They propose a benchmark dataset for evaluating the problem solving abilities of large language models (LLMs) they curate 515 challenging problems from the highly competitive IIT JEE-Advanced exam. |
| Outcome: | The proposed model performs better on open-source and proprietary models than the current model, but with techniques like self-consistency, self-refinement and chain-of-thought prompting. |