Papers with CLUTRR
Coupling Large Language Models with Logic Programming for Robust and General Reasoning from Text (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are robust and general, but their reasoning ability is not at a level to compete with the best models trained for specific natural language reasoning problems. |
| Approach: | They propose to use large language models as a few-shot semantic parser to convert natural language sentences into a logical form that serves as input for answer set programs. |
| Outcome: | The proposed model can handle multiple question-answering tasks without requiring retraining for each new task. |
CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text (D19-1)
Copied to clipboard
| Challenge: | Existing datasets for reading comprehension tasks have been used to test the generalization of natural language understanding systems. |
| Approach: | They propose a diagnostic benchmark suite to clarify key issues related to the robustness and systematicity of NLU systems. |
| Outcome: | The proposed benchmark suite clarifies key issues related to the robustness and systematicity of NLU systems. |