Papers by Osamu Imaichi
JFLD: A Japanese Benchmark for Deductive Reasoning Based on Formal Logic (2024.lrec-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have proficiently solved a broad range of tasks with their rich knowledge but struggle with logical reasoning. |
| Approach: | They propose a deductive reasoning benchmark for Japanese that assesses logical reasoning abilities isolated from knowledge and various reasoning rules. |
| Outcome: | The proposed benchmarks assess whether LLMs can generate logical steps to (dis)prove a given hypothesis based on a set of facts. |