Com2 : A Causal-Guided Benchmark for Exploring Complex Commonsense Reasoning in Large Language Models (2025.acl-long)
Copied to clipboard
Kai Xiong, Xiao Ding, Yixin Cao, Yuxiong Yan, Li Du, Yufei Zhang, Jinglong Gao, Jiaqian Liu, Bing Qin, Ting Liu
| Challenge: | Existing works focus on complex tasks like math and code, while complex commonsense reasoning remains underexplored due to its uncertainty and lack of structure. |
| Approach: | They propose to build a benchmark for large language models based on complex commonsense reasoning based upon causal event graphs and causal theory. |
| Outcome: | The proposed benchmark combines a complex commonsense reasoning benchmark with a detective story to achieve a more challenging subset. |
Similar Papers
Complex Reasoning over Logical Queries on Commonsense Knowledge Graphs (2024.acl-long)
Copied to clipboard
| Challenge: | Currently, language models struggle to generate commonsense inferences for complex tasks due to data scarcity and the difficulty of reasoning over multiple pieces of information. |
| Approach: | They propose a dataset to generate commonsense inferences from commonsensible data . they use a commonsence knowledge graph to extract and form questions from existing commonseense knowledge graphs. |
| Outcome: | The proposed dataset improves the ability of language models to reason about complex events without expensive human annotations. |
CausalGraph2LLM: Evaluating LLMs for Causal Queries (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have opened up new avenues for their use beyond standard Natural Language Processing tasks. |
| Approach: | They propose a benchmark to evaluate the capabilities of Large Language Models (LLMs) they use over 700k queries to compare their encoding capabilities. |
| Outcome: | The proposed benchmark compared LLMs on graph-level and node-level queries and open-sourced and closed models. |
CausalGraphBench: a Benchmark for Evaluating Language Models capabilities of Causal Graph discovery (2025.acl-srw)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have expanded their applications into domains not traditionally associated with natural language processing. |
| Approach: | They propose a benchmark to evaluate the ability of large language models to construct Causal Graphs (CGs) they examine various methods for CG discovery and their performance across different graph sizes and complexity levels. |
| Outcome: | The proposed benchmark comprises 35 CGs sourced from publicly available repositories and academic papers. |
On the Consistency of Commonsense in Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluations of commonsense for large language models focus on downstream knowledge tasks, failing to probe whether LLMs truly understand and utilize knowledge or merely memorize it. |
| Approach: | They propose to automatically construct a large benchmark named CoCo which measures LLMs’ knowledge memorization, comprehension, and application and examines the consistency between these tasks. |
| Outcome: | The proposed benchmark systematically assesses LLMs’ knowledge memorization, comprehension, and application and examines the consistency between these tasks. |
Causal-LLM: A Unified One-Shot Framework for Prompt- and Data-Driven Causal Graph Discovery (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current causal discovery methods rely on pairwise or iterative strategies that fail to capture global dependencies, amplify local biases, and reduce overall accuracy. |
| Approach: | They propose a framework for one-step full causal graph discovery using prompt-based discovery and a data-driven method for settings without metadata. |
| Outcome: | The proposed framework outperforms state-of-the-art models by approximately 40% in edge accuracy on datasets like Asia and Sachs while maintaining strong performance on more complex graphs. |
The Odyssey of Commonsense Causality: From Foundational Benchmarks to Cutting-Edge Reasoning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Despite its significance, a systematic exploration of commonsense causality is lacking. |
| Approach: | They focus on taxonomies, benchmarks, acquisition methods, qualitative reasoning, and quantitative measurements in commonsense causality. |
| Outcome: | The proposed method synthesizes insights from over 200 representative articles and provides a practical guide for beginners. |
METER: Evaluating Multi-Level Contextual Causal Reasoning in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks evaluate contextual causal reasoning in fragmented settings, failing to ensure context consistency or cover the full causal hierarchy. |
| Approach: | They use a unified context to benchmark large language models' contextual causal reasoning skills. |
| Outcome: | The proposed benchmarks show that LLMs are susceptible to distraction by irrelevant but factually correct information at lower level of causality. |
NoisyCausal: A Benchmark for Evaluating Causal Reasoning Under Structured Noise (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks focus on clean, abstract scenarios where causal structure is simple or implicitly assumed. |
| Approach: | They propose a benchmark to evaluate causal reasoning under structured noise. |
| Outcome: | The proposed method outperforms standard prompting and reasoning baselines on NoisyCausal. |
Causal Inference with Large Language Model: A Survey (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing causal inference frameworks do not match human judgment in several key areas, such as domain knowledge, logical inference, and cultural context. |
| Approach: | They propose to apply large language models to causal inference tasks . they summarize the main causal problems and approaches and compare their results . |
| Outcome: | The proposed methods are compared with traditional methods in healthcare, finance, and economics. |
Can Large Language Models Infer Causal Relationships from Real-World Text? (2026.acl-long)
Copied to clipboard
| Challenge: | Existing work evaluating large language models relies on synthetic or simplified texts with explicit causal relationships. |
| Approach: | They develop a benchmark to evaluate LLMs' ability to infer causal relationships from texts . they use a dataset of texts with different levels of explicitness and complexity . |
| Outcome: | The proposed benchmark is the first-ever real-world dataset for this task. |