Papers by Pardis Zahraei
WSC+: Enhancing The Winograd Schema Challenge Using Tree-of-Experts (2024.eacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) excel at answering WSC questions, but their ability to generate such questions remains less explored. |
| Approach: | They propose a tree-of-experts prompting method which enhances the generation of WSC instances by incorporating new 'ambiguous' and 'offensive' categories. |
| Outcome: | The proposed method enhances the generation of WSC instances (50% valid cases vs. 10% in recent methods) it extends the framework by incorporating new ‘ambiguous’ and ‘offensive’ categories, providing a deeper insight into model overconfidence and bias. |
TuringQ: Benchmarking AI Comprehension in Theory of Computation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | TuringQ is the first benchmark designed to evaluate the reasoning capabilities of large language models (LLMs) in the theory of computation. |
| Approach: | They propose a benchmark to evaluate the reasoning capabilities of large language models in the theory of computation. |
| Outcome: | The proposed system shows competitive accuracy when compared to human evaluation. |