Analyzing analytical methods: The case of phonology in neural models of spoken language (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies have focused on the strengths and weaknesses of various methods for analyzing phonology representations. |
| Approach: | They propose to use diagnostic classifiers and representational similarity analysis to quantify to what extent phonemes and phoneme sequences are encoded. |
| Outcome: | The proposed method is based on two commonly applied techniques . it shows that global-scope methods yield more consistent and interpretable results . |
Similar Papers
Interpretability and Analysis in Neural NLP (2020.acl-tutorials)
Copied to clipboard
| Challenge: | a tutorial aims to introduce the nascent field of interpretability and analysis of neural networks in NLP . |
| Approach: | This tutorial will introduce the nascent field of interpretability and analysis of neural networks in NLP. |
| Outcome: | This tutorial will introduce the nascent field of interpretability and analysis of neural networks in NLP. |
What Do Neural Speech Models Know About Phonology? Evidence from Structured Phoneme Confusions (2026.findings-acl)
Copied to clipboard
| Challenge: | acoustic and phonological models of speech recognition are often limited to the phoneme level . a recent study has shown that phoneme confusions are strongly structured in phonology space . |
| Approach: | They adopt a featural representation of phonemes grounded in phonological theory which models speech sounds as structured bundles of distinctive articulatory and acoustic properties. |
| Outcome: | The proposed model allows us to analyse phoneme confusions at a finer granularity and to investigate whether certain phonological features are more vulnerable than others. |
Computational Expressivity of Neural Language Models (2024.acl-tutorials)
Copied to clipboard
| Challenge: | Language models (LMs) are at the forefront of NLP research due to their versatility across diverse tasks. |
| Approach: | This tutorial will provide a framework for formal analysis of modern language models using tools from formal language theory. |
| Outcome: | This tutorial will provide a framework for formal analysis of modern language models using tools from formal language theory (FLT). |
Mapping Brains with Language Models: A Survey (2023.findings-acl)
Copied to clipboard
| Challenge: | accumulated evidence for brain and language model activations remains ambiguous, but correlations with model size and quality provide grounds for cautious optimism. |
| Approach: | They examine the evidence accumulated by 30 studies spanning 10 datasets and 8 metrics to determine whether there is any overlap between brain and language model activations. |
| Outcome: | The findings suggest that representations extracted from NLP models can (partially) explain the signal found in neural data. |
Correlating Neural and Symbolic Representations of Language (P19-1)
Copied to clipboard
| Challenge: | a popular technique for analyzing neural representations involves predicting information of interest from the activation patterns. |
| Approach: | They propose to use Representational Similarity Analysis and Tree Kernels to quantify how strongly activation patterns correspond to symbolic representations. |
| Outcome: | The proposed methods show that they exhibit the expected pattern of results on a synthetic language. |
The Grammar-Learning Trajectories of Neural Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | In this paper, we show that neural language models with different initialization, architecture, and training data acquire linguistic phenomena in a similar order, despite their different end performance. |
| Approach: | They propose to use mutual inductive bias to study linguistic representations implicit in NLMs. |
| Outcome: | The proposed approach shows that NLMs with different initialization, architecture, and training data acquire linguistic phenomena in a similar order, despite their different end performance. |
Do transformer models do phonology like a linguist? (2023.findings-acl)
Copied to clipboard
| Challenge: | Neural sequence-to-sequence models have been very successful at tasks in phonology and morphology that require a capacity for intricate linguistic generalisations. |
| Approach: | They propose to use a transformer model to model phonological generalisations on 29 different processes . they also use phonology as a primer and a rule with a withheld case to infer behaviour . |
| Outcome: | The proposed model can model all 29 phonological phenomena on a constructed language, regardless of perceived process difficulty. |
Neuron-level Interpretation of Deep NLP Models: A Survey (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing work on deep neural networks has focused on representation analysis, but recent work focused on analyzing neurons within these models. |
| Approach: | They propose to analyze neural networks to uncover linguistic concepts captured by the network . they propose to use a granular approach to analyze neurons within these models . |
| Outcome: | The proposed method combines methods to discover and understand neurons in a network with evaluation methods. |
Towards Explainable Evaluation of Language Models on the Semantic Similarity of Visual Concepts (2022.coling-1)
Copied to clipboard
Maria Lymperaiou, George Manoliadis, Orfeas Menis Mastromichalakis, Edmund G. Dervakos, Giorgos Stamou
| Challenge: | Recent advances in NLP research have focused on robustness and explainability issues of their evaluation strategies. |
| Approach: | They propose to use pre-trained transformers to evaluate semantic similarity for visual vocabularies . they propose to provide explainable metrics for understanding the quality of retrieved instances . |
| Outcome: | The proposed metrics highlight inabilities of widely used evaluation methods and highlight weaknesses in learned linguistic representations. |
Lexicosyntactic Inference in Neural Models (D18-1)
Copied to clipboard
| Challenge: | lexicosyntactic inferences are triggered by surprising aspects of the syntactical context that a word occurs in. |
| Approach: | They build a factuality judgment dataset for English clause-embedding verbs in various syntactic contexts and use it to probe the behavior of current state-of-the-art neural systems. |
| Outcome: | The proposed model makes systematic errors that are visible through the lens of factuality prediction. |