Papers by Gladys Tyen
Two Heads Are Better Than One: Dual-Model Verbal Reflection at Inference-Time (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing LLMs struggle to reliably detect subtle reasoning errors in ASAS tasks. |
| Approach: | They propose a dual-model framework with a dedicated Critic model trained for effective reflection that generates precise verbal feedback. |
| Outcome: | The proposed framework outperforms existing ASAS benchmarks and provides valuable insights into the performance of the proposed framework. |
BIG-Bench Extra Hard (2025.acl-long)
Copied to clipboard
Mehran Kazemi, Bahare Fatemi, Hritik Bansal, John Palowitch, Chrysovalantis Anastasiou, Sanket Vaibhav Mehta, Lalit K Jain, Virginia Aglietti, Disha Jindal, Peter Chen, Nishanth Dikkala, Gladys Tyen, Xin Liu, Uri Shalit, Silvia Chiappa, Kate Olszewska, Yi Tay, Vinh Q. Tran, Quoc V Le, Orhan Firat
| Challenge: | Current benchmarks for large language model reasoning focus on math and coding abilities, leaving a gap in evaluating broader reasoning proficiencies. |
| Approach: | They propose a benchmark to evaluate general reasoning in large language models . they use BIG-Bench and its harder version BIG-Benefit Hard to assess general reasoning . |
| Outcome: | The new benchmark pushes the boundaries of LLM reasoning evaluation. |
LLMs cannot find reasoning errors, but can correct them given the error location (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent attempts to self-correct logical or reasoning errors often cause correct answers to become incorrect, resulting in poor performance overall. |
| Approach: | They propose to use a backtracking setup to test the correction abilities of LLMs on their mistake-finding ability to find logical mistakes. |
| Outcome: | The proposed model improves on 5 reasoning tasks, showing that it can correct logical mistakes without ground truth labels or training data. |