Multimodal, Multilingual Grapheme-to-Phoneme Conversion for Low-Resource Languages (D19-61)
Copied to clipboard
| Challenge: | Grapheme-to-phoneme conversion (g2p) is a task of predicting the pronunciation of words from their orthographic representation. |
| Approach: | They propose to leverage audio data as an auxiliary modality in a multi-task training process to learn a more optimal grapheme representation. |
| Outcome: | The proposed model reduces phoneme error rate to 2.46% on in-domain test set compared to unimodal spelling- pronunciation model. |
Similar Papers
Zero-shot Learning for Grapheme to Phoneme Conversion with Language Ensemble (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing work focuses on low-resource and endangered languages with limited training sets. |
| Approach: | They propose a hypothesis set for any unseen target language and combine it with a confusion network to propose 'the most likely hypothesis' they test the approach on over 600 unseened languages and demonstrate it significantly outperforms baselines. |
| Outcome: | The proposed model outperforms baselines on over 600 unseen languages. |
GE2PE: Persian End-to-End Grapheme-to-Phoneme Conversion (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing text-to-speech systems struggle to produce natural speech from grapheme sequences . Grapheme-to phoneme conversion (G2P) systems face limitations when dealing with Persian texts due to the complexity of Persian transcription. |
| Approach: | They propose to use phonetic information to enhance the input sequence for Persian translations. |
| Outcome: | The proposed model surpasses state-of-the-art models by 1.86% in word error rate and 3.42% in homograph disambiguation accuracy. |
Supervised Grapheme-to-Phoneme Conversion of Orthographic Schwas in Hindi and Punjabi (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods to predict schwa deletion in Hindi are based on prosodic or phonetic analysis. |
| Approach: | They propose to use Hindi grapheme-to-phoneme (G2P) conversion to predict whether a schwa represented in the orthography is pronounced or unpronounced (deleted). |
| Outcome: | The proposed model outperforms existing models on a newly-compiled pronunciation lexicon extracted from various online dictionaries. |
Epitran: Precision G2P for Many Languages (L18-1)
Copied to clipboard
| Challenge: | Epitran is a multilingual, multi-back-end system for grapheme-to-phoneme transduction . it supports 61 languages and is open source under an MIT license . |
| Approach: | Epitran is a multilingual back-end system for grapheme-to-phoneme transduction . it takes word tokens in the orthography of a language and outputs a phonemic representation . Epitran's efficacy has been demonstrated in multiple research projects . |
| Outcome: | Epitran is a multilingual, multi-backend system for grapheme-to-phoneme transduction . it supports 61 languages and is open source under MIT license . |
A Two-Step Approach for Data-Efficient French Pronunciation Learning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have addressed intricate phonological phenomena in French, relying on extensive linguistic knowledge or a significant amount of sentence-level pronunciation data. |
| Approach: | They propose a grapheme-to-phoneme and post-lexical processing approach to address French phonological phenomena using sentence-level pronunciation data. |
| Outcome: | The proposed approach mitigates the lack of extensive labeled data and serves as a feasible solution for addressing French phonological phenomena even under resource-constrained environments. |
MultiBLiMP 1.0: A Massively Multilingual Benchmark of Linguistic Minimal Pairs (2026.tacl-1)
Copied to clipboard
| Challenge: | MultiBLiMP 1.0 is a massively multilingual benchmark of linguistic minimal pairs covering 101 languages and 2 types of subject-verb agreement. |
| Approach: | They propose to use multilingual benchmarks to evaluate linguistic minimal pairs in 101 languages and 2 types of subject-verb agreement to create the minimal pairs. |
| Outcome: | The proposed benchmark covers 101 languages and 2 types of subject-verb agreement, and contains more than 128,000 minimal pairs. |
Multimodality for NLP-Centered Applications: Resources, Advances and Frontiers (2022.lrec-1)
Copied to clipboard
| Challenge: | resurgence of multimodal datasets has attracted significant research interest, but there is no comprehensive survey for this task. |
| Approach: | They present a survey of a multimodal dataset with different modalities according to the applications. |
| Outcome: | The proposed datasets are available online and discuss the new frontier and motivate future researches. |
Multimodal neural pronunciation modeling for spoken languages with logographic origin (D18-1)
Copied to clipboard
| Challenge: | Graphemes of most languages encode pronunciation, though some are more explicit than others . pronunciation modeling in logographic languages requires decomposing logographs into subunits . |
| Approach: | They propose a multimodal approach to predict pronunciation of Cantonese logographic characters using neural networks. |
| Outcome: | The proposed framework improves performance by 18.1% and 25.0% respectively to unimodal and multimodal baselines. |
Multilingual-To-Multimodal (M2M): Unlocking New Languages with Monolingual Text (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing multimodal models rely on machine translation, but performance drops for other languages due to limited multilingual multimodal resources. |
| Approach: | They propose a lightweight alignment method that learns only a few linear layers using English text alone to map multilingual text embeddings into multimodal space. |
| Outcome: | M2M achieves strong zero-shot transfer on XTD Text-to-Image retrieval in English and spanish . it learns only a few linear layers to map multilingual text embeddings into multimodal space . |
GlotEval: A Test Suite for Massively Multilingual Evaluation of Large Language Models (2025.emnlp-demos)
Copied to clipboard
Hengyu Luo, Zihao Li, Joseph Attieh, Sawal Devkota, Ona de Gibert, Xu Huang, Shaoxiong Ji, Peiqin Lin, Bhavani Sai Praneeth Varma Mantina, Ananda Sreenidhi, Raúl Vázquez, Mengjie Wang, Samea Yusofi, Fei Yuan, Jörg Tiedemann
| Challenge: | Existing evaluation frameworks focus on English and a handful of high-resource languages, thereby overlooking the realistic performance of large language models in multilingual and lower-resourced scenarios. |
| Approach: | They propose a unified and lightweight framework that integrates 27 benchmarks under a standard ISO 639-3 language identifier system to enable seamless incorporation of new benchmarks. |
| Outcome: | The proposed framework integrates 27 benchmarks under a standard ISO 639-3 language identifier system, allowing for seamless incorporation of new benchmarks. |