Emergent morpho-phonological representations in self-supervised speech models (2025.emnlp-main)
Copied to clipboard
| Challenge: | a recent study shows that self-supervised speech models do not represent phonological and morphological phenomena in frequent English noun and verb inflections. |
| Approach: | They study how S3Ms represent phonological and morphological phenomena in English . they propose alternative representational strategies that may support human spoken word recognition . |
| Outcome: | a new study shows that S3M models can represent phonological and morphological phenomena in English . the models can be trained to recognize spoken words in naturalistic, noisy environments . |
Similar Papers
[b] = [d] - [t] + [p]: Self-supervised Speech Models Discover Phonological Vector Arithmetic (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on how self-supervised speech models encode rich phonetic information have not explored how they are structured. |
| Approach: | They conduct a comprehensive analysis of the underlying structure of S3M representations with particular attention to phonological vectors. |
| Outcome: | The proposed model encodes phonologically interpretable and compositional vectors, demonstrating phonology vector arithmetic. |
Layer-wise Minimal Pair Probing Reveals Contextual Grammatical-Conceptual Hierarchy in Speech Representations (2025.emnlp-main)
Copied to clipboard
| Challenge: | a recent study evaluated the extent to which SLMs encode nuanced syntactic and conceptual features . acoustic and phonetic features are shallow, but the extent of nuance is unclear . |
| Approach: | a new study evaluates contextual syntactic and semantic features in transformer-based speech language models . authors compare SLMs to linguistic competence assessments for large language models. |
| Outcome: | a new study compares SLMs with linguistic competence assessments to assess speech recognition and understanding . the results show that SLM models encode grammatical features more robustly than conceptual ones . |
Do self-supervised speech models develop human-like perception biases? (2022.acl-long)
Copied to clipboard
| Challenge: | Recent advances in speech recognition and representation learning show that self-supervised pretraining is an excellent way of improving performance while reducing the amount of labelled data needed for training. |
| Approach: | They compare the representational spaces of wav2vec, HuBERT and contrastive predictive coding (CPC) with the perceptual spaces of French-speaking and English-speaking human listeners. |
| Outcome: | The proposed models capture fine-grained perceptual phenomena while supervised models are better at capturing coarser, phone-level effects and effects of listeners’ native language on perception. |
Unlike “Likely”, “Unlike” is Unlikely: BPE-based Segmentation hurts Morphological Derivations in LLMs (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) use subword vocabularies to process and generate text. |
| Approach: | They find that Large Language Models (LLMs) perform poorly at handling some types of affixations because subwords are marked as initial- or intra-word . |
| Outcome: | The largest models trained on enough data can mitigate this tendency because initial- and intra-word embeddings are aligned; in-context learning also helps when all examples are selected in a consistent way; but only morphological segmentation can achieve a near-perfect accuracy. |
Dissecting Contextual Word Embeddings: Architecture and Representation (D18-1)
Copied to clipboard
| Challenge: | Existing work on learning contextual representations has used LSTM-based biLMs, but there is no reason to believe this is effective. |
| Approach: | They propose to use pre-trained bidirectional language models to learn contextual word embeddings for four NLP tasks and to use them to study the effects of architecture on endtask accuracy. |
| Outcome: | The proposed models outperform word embeddings for four NLP tasks and all learn representations that vary with network depth. |
Exploring Layer-wise Representations of English and Chinese Homonymy in Pre-trained Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | lexical ambiguity can arise due to the misunderstanding of its multiple senses. |
| Approach: | They propose to use part of speech to examine homonyms in Chinese and English . they find no universal layer depth excels in differentiating homnomial representations . |
| Outcome: | The proposed model improves contextualization of homonym representations in Chinese . the results challenge the simplistic understanding of their inner workings, the authors say . |
Radical Allomorphy: Phonological Surface Forms without Phonology (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work typically frames morphophonology as generating surface forms from abstract underlying representations (URs) this theory-laden assumption is expensive to annotate, especially in low-resource settings. |
| Approach: | a new approach frames morphophonology as generating surface forms from abstract underlying representations by applying phonological rules or constraints. |
| Outcome: | The proposed model removes the need to posit or label URs and lets the model exploit the surface evidence directly. |
Morphological Inflection with Phonological Features (2023.acl-short)
Copied to clipboard
| Challenge: | Recent advances in morphological tasks can be difficult to solve when little training data is available or when generalizing to previously unseen lemmas. |
| Approach: | They propose two methods to manipulate phonemic data to include phonological features instead of characters. |
| Outcome: | The proposed methods yield comparable results to baseline models, with minor improvements in some languages. |
Exploring Linguistic Probes for Morphological Inflection (2023.emnlp-main)
Copied to clipboard
| Challenge: | morphological inflection models typically employ language-independent data splitting algorithms. |
| Approach: | They propose language-specific probes to test aspects of morphological generalization . they use three morphology-distinct languages to test their generalization abilities . |
| Outcome: | The proposed language-specific probes are used to test morphological generalization abilities on three distinct languages. |
Encoding of lexical tone in self-supervised models of spoken language (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing research on representations of phonetic and phonological information has focused on segmental features such as phonemes. |
| Approach: | They propose to analyze the tone encoding capabilities of self-supervised Spoken Language Models, using Mandarin and Vietnamese as case studies. |
| Outcome: | The proposed models encode lexical tone even when trained on non-tonal languages. |