Papers by Zhaoqi Liu
ChessArena: A Chess Testbed for Evaluating Strategic Reasoning Capabilities of Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation frameworks focus on isolated question-answering tasks that may not capture the essential aspects of strategic reasoning. |
| Approach: | They evaluate 13 large language models across over 800 games in chess . they use a chessian-based framework to test strategic reasoning and pattern recognition . |
| Outcome: | The proposed framework improves performance and basic understanding of large language models. |
Semantic Sensitivities and Inconsistent Predictions: Measuring the Fragility of NLI Models (2024.eacl-long)
Copied to clipboard
| Challenge: | Recent studies have indicated that NLI models have an understanding of lexical and compositional semantics. |
| Approach: | They propose a framework to assess the extent of semantic sensitivity in NLI models . they use adversarially generated examples with minor semantics-preserving surface-form variations . |
| Outcome: | The proposed framework shows that NLI models struggle with minor variations requiring knowledge of compositional semantics . |