Papers by Zeyi Liao
RobustLR: A Diagnostic Benchmark for Evaluating Logical Robustness of Deductive Reasoners (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that perform deductive reasoning on inputs containing rules and statements in the English natural language do not perform consistently on the RobustLR test set. |
| Approach: | They propose a diagnostic benchmark that evaluates the robustness of language models to minimal logical edits in inputs and different logical equivalence conditions. |
| Outcome: | The proposed models do not perform consistently on the RobustLR test set. |
AttributionBench: How Hard is Automatic Attribution Evaluation? (2024.findings-acl)
Copied to clipboard
| Challenge: | generative search engines enhance the reliability of large language model responses by providing cited evidence. |
| Approach: | They propose to use a benchmark to evaluate whether a large language model supports the generated responses or not . |
| Outcome: | The proposed benchmark shows that even a fine-tuned GPT-3.5 only achieves around 80% macro-F1 under a binary classification formulation. |
In Search of the Long-Tail: Systematic Generation of Long-Tail Inferential Knowledge via Logical Rule Guided Search (2024.emnlp-main)
Copied to clipboard
Huihan Li, Yuting Ning, Zeyi Liao, Siyuan Wang, Xiang Li, Ximing Lu, Wenting Zhao, Faeze Brahman, Yejin Choi, Xiang Ren
| Challenge: | Logic-Induced-Knowledge-Search (LINK) is a framework for generating factually-correct yet long-tail inferential knowledge. |
| Approach: | They introduce a framework to obtain factually-correct yet long-tail inferential statements using variable-wise prompting grounded on symbolic rules. |
| Outcome: | The proposed framework is able to obtain factually-correct yet long-tail inferential statements while ensuring factual correctness. |