Morphosyntactic Tagging with Pre-trained Language Models for Arabic and its Dialects (2022.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained morphosyntactic tagging models outperform existing systems in Modern Standard Arabic and all the Arabic dialects studied. |
| Approach: | They present results on morphosyntactic tagging across different varieties of Arabic using pre-trained transformer language models. |
| Outcome: | The proposed models outperform existing systems in Modern Standard Arabic, 2.8% in Gulf, 1.6% in Egyptian, and 8.3% in Levantine. |
Similar Papers
Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language Processing (2022.emnlp-main)
Copied to clipboard
Abbas Ghaddar, Yimeng Wu, Sunyam Bagga, Ahmad Rashid, Khalil Bibi, Mehdi Rezagholizadeh, Chao Xing, Yasheng Wang, Xinyu Duan, Zhefeng Wang, Baoxing Huai, Xin Jiang, Qun Liu, Phillippe Langlais
| Challenge: | Existing pre-trained language models are not well-explored and are not reproducible in the literature. |
| Approach: | They propose to improve existing Arabic language pre-trained language models using a more methodical approach. |
| Outcome: | The proposed models outperform existing models on ALUE, a leaderboard-powered benchmark for Arabic NLU and NLG tasks. |
Self-Training Pre-Trained Language Models for Zero- and Few-Shot Multi-Dialectal Arabic Sequence Labeling (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches to fine-tune pre-trained language models for downstream tasks require labeled data. |
| Approach: | They propose to self-train pre-trained language models to improve performance on data-scarce varieties by as large as 10% F1 and 2% accuracy. |
| Outcome: | The proposed model improves zero-shot MSA-to-DA transfer by as large as 10% F1 (NER) and 2% accuracy (POS tagging). |
Multi-Dialect Arabic POS Tagging: A CRF Approach (L18-1)
Copied to clipboard
Kareem Darwish, Hamdy Mubarak, Ahmed Abdelali, Mohamed Eldesouki, Younes Samih, Randah Alharbi, Mohammed Attia, Walid Magdy, Laura Kallmeyer
| Challenge: | Existing work on dialectal POS tagging is rather scant with POS tags for most dialects being nonexistent or of limited availability. |
| Approach: | They propose a dataset of POS-tagged Arabic tweets in four major dialects and a tagging guideline for each dialect. |
| Outcome: | The proposed model can tag four different dialects with an average accuracy of 89.3%. |
Arabic Dialect Identification with a Few Labeled Examples Using Generative Adversarial Networks (2022.aacl-main)
Copied to clipboard
| Challenge: | Experimental results show that transformer-based models can handle Dialect Arabic (DA) classification tasks with a large corpus of labeled examples. |
| Approach: | They extend transformer-based models with unlabeled data in a generative adversarial setting using semi-supervised Generative Adversarial Networks (SS-GAN) they show that the model can produce high-quality embeddings for the Dialect Arabic examples and generalize for the downstream classification task given few labeled examples. |
| Outcome: | The proposed model outperforms models with unlabeled data in a generative adversarial setting with unlabelled examples and faster convergence when only a few labeled examples are available. |
Fine-grained Morphosyntactic Analysis and Generation Tools for More Than One Thousand Languages (2020.lrec-1)
Copied to clipboard
| Challenge: | Using morphosyntactic tools, we train and distribute tools for approximately one thousand languages. |
| Approach: | They train and distribute morphosyntactic tools for approximately one thousand languages. |
| Outcome: | The results show that the tools generalize well across rare and common forms alike. |
ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic (2021.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models (LMs) are expensive and limited in inference time . a new benchmark for multi-dialectal Arabic language understanding evaluation is developed . |
| Approach: | They introduce two powerful deep bidirectional transformer-based models, ARBERT and MARBERT . they also introduce ARLUE, a new benchmark for multi-dialectal Arabic language understanding evaluation . |
| Outcome: | The proposed models outperform monolingual models with larger vocabulary and larger datasets in Arabic language understanding evaluation. |
AraT5: Text-to-Text Transformers for Arabic Language Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing models that convert text-based language problems into text-to-text format are not suitable for multilingual tasks. |
| Approach: | They propose a unified Transformer framework that converts all language problems into a text-to-text format. |
| Outcome: | The proposed model performs better on all ARGEN tasks than existing models with 49 less data. |
ArabicTransformer: Efficient Large Arabic Language Model with Funnel Transformer and ELECTRA Objective (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing solutions to reduce the cost of pretraining Transformer-based models are expensive especially for large-scale models. |
| Approach: | They propose to reduce the cost of pre-training Transformer-based models by compressing the sequence of hidden states inside Transformer architecture. |
| Outcome: | The proposed model achieves state-of-the-art on several Arabic downstream tasks despite using less computational resources compared to other BERT-based models. |
Challenging the Transformer-based models with a Classical Arabic dataset: Quran and Hadith (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing benchmark datasets have a low readability index which does not reflect real-world complex data. |
| Approach: | They constructed a dataset of Quran-verse and Hadith-teaching pairs by consulting sources of reputable religious experts. |
| Outcome: | The proposed models performed on a binary classification task to identify whether two pieces of CA text convey the same underlying message. |
Noise-Robust Morphological Disambiguation for Dialectal Arabic (N18-1)
Copied to clipboard
| Challenge: | Noisy content is non-canonical in nature, with lexical, orthographic, and phonetic variations. |
| Approach: | They propose a neural morphological tagging and disambiguation model for Egyptian Arabic with various extensions to handle noisy content. |
| Outcome: | The proposed model achieves about 5% relative error reduction over a state-of-the-art baseline for Egyptian Arabic. |