Papers by Xianting Huang
Breaking the Evaluation Paradox: Evaluating High-Entropy Search with Computationally Irreducible Constraints (2026.findings-acl)
Copied to clipboard
| Challenge: | a new framework for evaluation of exhaustive search capabilities is needed . high-entropy enumeration tasks make such ground truth impossible for humans to create . VERITAS is a framework built on the principle of computationally irreducible constraints . |
| Approach: | They propose a framework that uses non-optimizable constraints to create verifiable searches . VERITAS can generate infinite number of test cases with perfect ground truth and precise difficulty control . |
| Outcome: | a new evaluation framework for large language models is based on non-optimizable constraints . the framework can generate infinite number of test cases with perfect ground truth and precise difficulty control . |