Using Shapley interactions to understand how models use structure (2025.acl-long)
Copied to clipboard
| Challenge: | Language models are intricately structured systems, and attribution measures are important for understanding how they combine features to influence outputs. |
| Approach: | They use Shapley Taylor interaction indices to examine how language and speech models internally relate and structure their inputs. |
| Outcome: | The proposed methods show that language models encode phonetic interactions . they show that the inputs are more entangled for pairs where a consonant influences a vowel or approximant . |
Similar Papers
Feature Interactions Reveal Linguistic Structure in Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing features attribution methods for post-hoc interpretability ignore the existence of interactions between the effects of features on the prediction. |
| Approach: | They propose a grey box method to train models to perfection on a formal language classification task using PCFGs. |
| Outcome: | The proposed methods are able to uncover the grammatical rules acquired by the model under specific configurations and provide novel insights into the linguistic structure of the target models. |
Incorporating Contextual and Syntactic Structures Improves Semantic Similarity Modeling (D19-1)
Copied to clipboard
| Challenge: | Semantic similarity modeling is central to many NLP problems such as question answering. |
| Approach: | They propose a pairwise word interaction model with syntactic structure priors to explore their effectiveness. |
| Outcome: | Extensive evaluations on eight benchmark datasets show that incorporating structural information improves over strong baselines. |
Linguistically Grounded Analysis of Language Models using Shapley Head Values (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for probing language models for morphosyntactic constructions are not well understood . language models gain knowledge of grammatical phenomena during pretraining, but exactly how this knowledge is encoded is not well established. |
| Approach: | They propose a method for probing language models via Shapley Head Values . they use a BLiMP dataset to test their method on linguistic constructions based on a Shaply Head Value method . |
| Outcome: | The proposed method can be used to investigate linguistic knowledge in language models . it shows that attention heads responsible for processing related linguistic phenomena cluster together . |
Understanding the Inner-workings of Language Models Through Representation Dissimilarity (2023.emnlp-main)
Copied to clipboard
| Challenge: | Dissimilarity measures measure the extent to which two model’s internal representations differ . they can identify and locate generalization properties of models that are invisible via in-distribution test set performance. |
| Approach: | They propose to use representation dissimilarity measures to measure the extent to which two model’s internal representations differ. |
| Outcome: | The proposed dissimilarity measures can identify and locate generalization properties of models that are invisible via in-distribution test set performance and new evaluations of how language model features vary as width and depth are increased. |
Overestimation of Syntactic Representation in Neural Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Several testing methodologies have been developed to probe models’ syntactic representations. |
| Approach: | They propose a method to determine syntactic structure by training a model on strings generated according to a template and testing its ability to distinguish between similar ones with different syntax. |
| Outcome: | The proposed method reproduces positive results with two non-syntactic baseline language models: an n-gram model and an LSTM model trained on scrambled inputs. |
Exploiting Syntactic Structure for Better Language Modeling: A Syntactic Distance Approach (2020.acl-main)
Copied to clipboard
| Challenge: | incorporating syntactic structure into language models has been a challenge since the 1990s. |
| Approach: | They propose to use syntactic information to integrate syntastic structure into neural language models by providing ground truth parse trees as additional training signals. |
| Outcome: | The proposed model achieves lower perplexity and better quality when ground truth parse trees are provided as training signals. |
Explaining Pre-Trained Language Models with Attribution Scores: An Analysis in Low-Resource Settings (2024.lrec-main)
Copied to clipboard
| Challenge: | Currently, prompt-based models are gaining popularity due to their easier adaptability in low-resource settings. |
| Approach: | They analyze attribution scores extracted from prompt-based models w.r.t. plausibility and faithfulness and compare them with attribution score extracted from fine-tuned models and large language models. |
| Outcome: | The proposed model outperforms attention and Integrated Gradients in plausibility and faithfulness, while fine-tuning models are harder to explain in low-resource settings. |
LSTMs Compose—and Learn—Bottom-Up (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work in NLP shows that LSTMs capture compositional structure in language data. |
| Approach: | They propose to measure the decompositional interdependence between word meanings in an LSTM based on their gate interactions. |
| Outcome: | The proposed model can model syntactic relationships rather than learning the longer-range relations independently. |
What Does BERT Learn about the Structure of Language? (P19-1)
Copied to clipboard
| Challenge: | BERT is a language representation model that has performed well in diverse language understanding benchmarks. |
| Approach: | They perform experiments to unpack the elements of English language structure learned by BERT. |
| Outcome: | The proposed model outperforms state-of-the-art models in the GLUE benchmark by a significant margin. |
Do Language Models Exhibit Human-like Structural Priming Effects? (2024.findings-acl)
Copied to clipboard
| Challenge: | a recent exposure to a structure facilitates processing of the same structure, a study finds . structural priming is well attested in humans, for both language production and comprehension . |
| Approach: | They use the structural priming paradigm to investigate where priming effects manifest . they find that rarer elements within a prime increase priming effect . |
| Outcome: | The findings provide an important piece in the puzzle of understanding how properties within their context affect structural prediction in language models. |