| Challenge: | Despite the scaling of causal language models, the underlying basis of complex skills remains unclear. |
| Approach: | They propose that subjacent skills such as synonymy might be explained using linguistic concepts. |
| Outcome: | The proposed model recognizes synonymy but struggles to generate synonyms when prompted with relevant context. |
Similar Papers
Can Post-Training Transform LLMs into Causal Reasoners? (2026.findings-acl)
Copied to clipboard
| Challenge: | Causal inference is a core component of human cognition and requires decision-makers to distinguish between causation and association. |
| Approach: | They propose a dataset comprising seven core causal tasks for training and five diverse test sets and evaluate five different post-training approaches. |
| Outcome: | The proposed model achieves 93.5% accuracy on the CaLM benchmark, compared to 55.4% by OpenAI o3. |
On the Reliability of Large Language Models for Causal Discovery (2025.acl-long)
Copied to clipboard
| Challenge: | Existing statistical methods to identify causal relationships from observational data remain elusive. |
| Approach: | They examine the impact of memorization for accurate causal relation prediction, the influence of incorrect causal relations in pre-training data and the contextual nuances that influence LLMs’ understanding of causal relations. |
| Outcome: | The proposed models are effective in recognizing causal relations that occur frequently in pre-training data, but their ability to generalize to new or rare causal relations is limited. |
Large Language Models and Causal Inference in Collaboration: A Comprehensive Survey (2025.findings-naacl)
Copied to clipboard
Xiaoyu Liu, Paiheng Xu, Junda Wu, Jiaxin Yuan, Yifan Yang, Yuhang Zhou, Fuxiao Liu, Tianrui Guan, Haoliang Wang, Tong Yu, Julian McAuley, Wei Ai, Furong Huang
| Challenge: | Large Language Models (LLMs) have shown great potential to enhance Natural Language Processing (NLP) models in areas such as predictive accuracy, fairness, robustness, and explainability. |
| Approach: | They evaluate or improve generative Large Language Models from a causal perspective in areas such as reasoning capacity, fairness and safety issues, explainability, and handling multimodality. |
| Outcome: | The proposed models can be used to perform causal relationship discovery and causal effect estimation tasks. |
Can Large Language Models Learn Independent Causal Mechanisms? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) perform poorly on complex reasoning tasks, such as abstract, causal, or logical reasoning. |
| Approach: | They propose to use two concepts from causality to learn ICMs within LLMs to improve out-of-distribution performance on abstract and causal reasoning tasks. |
| Outcome: | The proposed model outperforms existing models on abstract and causal reasoning tasks and is more robust to fine-tuning. |
Causal Inference with Large Language Model: A Survey (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing causal inference frameworks do not match human judgment in several key areas, such as domain knowledge, logical inference, and cultural context. |
| Approach: | They propose to apply large language models to causal inference tasks . they summarize the main causal problems and approaches and compare their results . |
| Outcome: | The proposed methods are compared with traditional methods in healthcare, finance, and economics. |
ContraCLM: Contrastive Learning For Causal Language Model (2023.acl-long)
Copied to clipboard
Nihal Jain, Dejiao Zhang, Wasi Uddin Ahmad, Zijian Wang, Feng Nan, Xiaopeng Li, Ming Tan, Ramesh Nallapati, Baishakhi Ray, Parminder Bhatia, Xiaofei Ma, Bing Xiang
| Challenge: | Existing studies show that causal language models lack expressiveness due to poor discrimination ability. |
| Approach: | They propose a contrastive learning framework that enhances discrimination of representations and bridges the gap with encoder-only models. |
| Outcome: | The proposed framework improves discrimination and source code generation capabilities on a variety of downstream tasks. |
Lexical Popularity: Quantifying the Impact of Pre-training for LLM Performance (2026.eacl-long)
Copied to clipboard
| Challenge: | Large Language Models excel in varied tasks, but their mechanisms remain unclear . current LLMs' development has put this assumption in jeopardy, authors say . |
| Approach: | They examine whether LLMs learn generalized linguistic abstraction or rely on surface-level features that match their pre-training data. |
| Outcome: | The proposed model can learn generalized linguistic abstraction or rely on surface-level features that match their pre-training data. |
CausalEval: Towards Better Causal Reasoning in Language Models (2025.naacl-long)
Copied to clipboard
Longxuan Yu, Delin Chen, Siheng Xiong, Qingyang Wu, Dawei Li, Zhikai Chen, Xiaoze Liu, Liangming Pan
| Challenge: | Large language models (LLMs) have been used for a variety of tasks, including problem-solving, decision-making, and understanding of the world. |
| Approach: | They propose a review of existing methods aimed at enhancing LMs for causal reasoning . they categorize existing methods as reasoning engines or as helpers providing knowledge or data to traditional methods . |
| Outcome: | The proposed methods perform better than existing methods on a range of tasks. |
Probing Pretrained Language Models for Lexical Semantics (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have focused on morphosyntactic, semantic, and world knowledge, but it remains unclear to what extent LMs derive lexical type-level knowledge from words in context. |
| Approach: | They propose to use multilingual and monolingual LMs to extract lexical type-level knowledge from words in context. |
| Outcome: | The proposed models perform well across six typologically diverse languages and five lexical tasks. |
Exploring the Representation of Word Meanings in Context: A Case Study on Homonymy and Synonymy (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models that represent different senses of words in context are not accurate for polysemous words. |
| Approach: | They propose a multilingual dataset that evaluates the ability of models to accurately represent different lexical-semantic relations such as homonymy and synonymy. |
| Outcome: | The proposed models can disambiguate homonyms in context, but fail to represent words with different senses when occurring in similar sentences. |