Morfessor EM+Prune: Improved Subword Segmentation with Expectation Maximization and Pruning (2020.lrec-1)
Copied to clipboard
| Challenge: | Subword segmentation is a standard preprocessing step in many neural approaches to natural language processing. |
| Approach: | They propose to train a unigram subword model using a recursive algorithm and lexicon pruning algorithm. |
| Outcome: | The proposed method improves on the original training algorithm and improves morphological segmentation accuracy. |
Similar Papers
Lexically Grounded Subword Segmentation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Statistical word segmentation algorithms have remained a thorn in the side of many researchers. |
| Approach: | They propose to use unsupervised morphological analysis with Morfessor as pre-tokenization and an algebraic method for obtaining subword embeddings grounded in a word embeddable space. |
| Outcome: | The proposed methods improve morphological plausibility and Rényi efficiency on part-of-speech tagging and machine translation tasks. |
How Suitable Are Subword Segmentation Strategies for Translating Non-Concatenative Morphology? (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Data-driven subword segmentation is the default strategy for open-vocabulary machine translation but may not be sufficiently generic for learning non-concatenative morphology. |
| Approach: | They propose to test data-driven subword segmentation on non-concatenative morphological phenomena in a controlled, semi-synthetic setting. |
| Outcome: | The proposed model can translate non-concatenative morphological phenomena in a controlled, semi-synthetic setting. |
Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates (P18-1)
Copied to clipboard
| Challenge: | Subword units are an effective way to alleviate the open vocabulary problems in neural machine translation. |
| Approach: | They propose a method to regularize subword segmentations probabilistically by sampling subwords . they also propose 'unigram' language model to be used for better subword sampling . |
| Outcome: | The proposed method improves on low resource and out-of-domain settings with multiple corpora. |
When is Char Better Than Subword: A Systematic Study of Segmentation Algorithms for Neural Machine Translation (2021.acl-short)
Copied to clipboard
| Challenge: | Subword segmentation algorithms can produce sub-optimal segmentation when the target language is rich in morphological changes or there is not enough data for learning compact composition rules. |
| Approach: | They compare character-based and subword-based neural machine translation systems . they find character-driven models are better at handling morphological phenomena . |
| Outcome: | The character-based models are better at handling morphological phenomena, generating rare and unknown words, and more suitable for transferring to unseen domains. |
LLMSegm: Surface-level Morphological Segmentation Using Large Language Model (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to morphological segmentation split word into its morphemes . LLMSegm is applicable in low-data settings and low-resourced languages . |
| Approach: | They propose a novel approach to surface-level morphological segmentation leveraging large language models. |
| Outcome: | The proposed method is applicable in low-data settings and low-resource languages. |
Subword Segmental Machine Translation: Unifying Segmentation and Target Sentence Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Subword segmenters are used in neural machine translation, but are not used in high-resource settings. |
| Approach: | They propose a subword segmental machine translation (SSMT) that unifies subword and MT in a single trainable model. |
| Outcome: | The proposed model improves chrF scores for morphologically rich agglutinative languages and is more robust on a test set constructed for evaluating morphology generalisations. |
BERTSeg: BERT Based Unsupervised Subword Segmentation for Neural Machine Translation (2022.aacl-short)
Copied to clipboard
| Challenge: | Existing subword segmenters are frequency-based without semantics information or neural-based but trained on parallel corpora. |
| Approach: | They propose an unsupervised neural subword segmenter for neural machine translation that utilizes contextualized semantic embeddings of words from characterBERT and maximizes the generation probability of subword segments. |
| Outcome: | The proposed method improves translation performance on ALT, IWSLT15 Vi->En, WMT16 Ro->En and WMT15 Fi->En datasets. |
The Effectiveness of Morphology-aware Segmentation in Low-Resource Neural Machine Translation (2021.eacl-srw)
Copied to clipboard
| Challenge: | Current NMT systems typically operate at the level of subwords, causing problems of vocabulary sparsity. |
| Approach: | They compare subword segmentation methods with morphologically-based methods in a low-resource setting . they find that no consistent and reliable differences emerge between the methods . |
| Outcome: | The proposed methods outperform BPE in a low-resource translation setting. |
Unlike “Likely”, “Unlike” is Unlikely: BPE-based Segmentation hurts Morphological Derivations in LLMs (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) use subword vocabularies to process and generate text. |
| Approach: | They find that Large Language Models (LLMs) perform poorly at handling some types of affixations because subwords are marked as initial- or intra-word . |
| Outcome: | The largest models trained on enough data can mitigate this tendency because initial- and intra-word embeddings are aligned; in-context learning also helps when all examples are selected in a consistent way; but only morphological segmentation can achieve a near-perfect accuracy. |
Subword Segmental Language Modelling for Nguni Languages (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Subword segmentation is a standard practice in NLP, but is viewed as a preprocessing step for low-resource languages with complex morphologies. |
| Approach: | They propose a subword segmental language model that learns how to segment words while being trained for autoregressive language modelling. |
| Outcome: | The proposed model outperforms existing models on unsupervised morphological segmentation and outperfies standard subword segmenters on all 4 languages. |