| Challenge: | Existing methods for probing are limited and lack understanding of their limitations and weaknesses. |
| Approach: | They propose a strategy for input-level intervention on naturalistic sentences . they use morpho-syntactic features of a sentence to intervene on the rest of the sentence . |
| Outcome: | The proposed approach allows for input-level intervention on naturalistic sentences while keeping the rest of the sentence unchanged. |
Similar Papers
Interventional Probing in High Dimensions: An NLI Case Study (2023.findings-eacl)
Copied to clipboard
| Challenge: | Probing strategies have been shown to detect the presence of various linguistic features inlarge language models; in particular, semantic features intermediate to the “natural logic”fragment of the NLI. |
| Approach: | They propose to use amnesic probing and mnestic probing to investigate the effect of these semantic fea-tures on NLI classification by examining the effects of a mnemonic probing variation on the model. |
| Outcome: | The proposed methods have been shown to detect features intermediate to the “natural logic”fragment of the Natural Language Inferencetask (NLI). |
A multilabel approach to morphosyntactic probing (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Morphologically rich languages present unique challenges to natural language processing . morphological supervision can improve the quality of multilingual language models . |
| Approach: | They propose a multilabel probing task to assess morphosyntactic representations of multilingual word embeddings. |
| Outcome: | The proposed probing task makes it easy to explore morphosyntactic representations . it also allows the study of how language models handle co-occurring features . |
Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond (2022.tacl-1)
Copied to clipboard
Amir Feder, Katherine A. Keith, Emaad Manzoor, Reid Pryzant, Dhanya Sridhar, Zach Wood-Doughty, Jacob Eisenstein, Justin Grimmer, Roi Reichart, Margaret E. Roberts, Brandon M. Stewart, Victor Veitch, Diyi Yang
| Challenge: | causality has not had the same importance in natural language processing, says aaron e. smith . he says research on causality in NLP remains scattered across domains without unified definitions . |
| Approach: | They propose to consolidate research on causality in NLP across academic areas . they explore potential uses of causal inference to improve robustness, fairness, interpretability . |
| Outcome: | The proposed method is a unified overview of causal inference for the NLP community. |
Large Language Models and Causal Inference in Collaboration: A Comprehensive Survey (2025.findings-naacl)
Copied to clipboard
Xiaoyu Liu, Paiheng Xu, Junda Wu, Jiaxin Yuan, Yifan Yang, Yuhang Zhou, Fuxiao Liu, Tianrui Guan, Haoliang Wang, Tong Yu, Julian McAuley, Wei Ai, Furong Huang
| Challenge: | Large Language Models (LLMs) have shown great potential to enhance Natural Language Processing (NLP) models in areas such as predictive accuracy, fairness, robustness, and explainability. |
| Approach: | They evaluate or improve generative Large Language Models from a causal perspective in areas such as reasoning capacity, fairness and safety issues, explainability, and handling multimodality. |
| Outcome: | The proposed models can be used to perform causal relationship discovery and causal effect estimation tasks. |
Do Syntactic Probes Probe Syntax? Experiments with Jabberwocky Probing (2021.naacl-main)
Copied to clipboard
| Challenge: | a study of neural language models shows that syntactic probes do not properly isolate syntax. |
| Approach: | They show that syntactic probes do not properly isolate syntax . they train two probes trained on normal data and find they perform worse . |
| Outcome: | The proposed method outperforms the baseline models on the most popular models, but their lead is reduced by 53%. |
MorphNLI: A Stepwise Approach to Natural Language Inference Using Text Morphing (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing models fail to capture important semantic features of logic such as monotonicity and negation. |
| Approach: | They propose a modular step-by-step approach to natural language inference . they use a language model to generate edits to incrementally transform the premise into the hypothesis . |
| Outcome: | The proposed method outperforms baseline models in realistic cross-domain settings with improvements up to 12.6% (relative). |
Can Prompt Probe Pretrained Language Models? Understanding the Invisible Risks from a Causal View (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies have found prompt-based probing evaluations inaccurate, inconsistent and unreliable. |
| Approach: | They propose to conduct debiasing via causal intervention to uncover biases in probing evaluations . authors argue that prompt-based probing is inaccurate, inconsistent and unreliable . |
| Outcome: | This paper examines the effectiveness of prompt-based probing in pretrained language models . it highlights critical biases which could induce biased results and conclusions . authors suggest rethinking criteria for evaluating better pretrained models based on such evaluations . |
The Functional Relevance of Probed Information: A Case Study (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent studies have shown that transformer models like BERT rely on number information encoded in their representations of sentences’ subjects and head verbs when performing subject-verb agreement. |
| Approach: | They propose to use probing to find out which words contain functionally relevant information encoded in the representations of subject plurality and words that agree with it in number in BERT. |
| Outcome: | The proposed model only uses the subject plurality information encoded in its representations of the subject and words that agree with it in number. |
When Does Syntax Mediate Neural Language Model Performance? Evidence from Dropout Probes (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that models encode syntactic information redundantly . this allows researchers to boost models' performance by injecting syntaktic information into embeddings . |
| Approach: | They propose a new probe design that guides probes to consider all syntactic information present in embeddings. |
| Outcome: | The proposed model improves performance by injecting syntactic information into models. |
Causal Inference with Large Language Model: A Survey (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing causal inference frameworks do not match human judgment in several key areas, such as domain knowledge, logical inference, and cultural context. |
| Approach: | They propose to apply large language models to causal inference tasks . they summarize the main causal problems and approaches and compare their results . |
| Outcome: | The proposed methods are compared with traditional methods in healthcare, finance, and economics. |