ViDeBERTa: A powerful pre-trained language model for Vietnamese (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing models for Vietnamese that perform well on downstream tasks, such as Question answering, are based on Transformer. |
| Approach: | They propose a pre-trained monolingual Vietnamese model with three versions . they fine-tune and evaluate the model on three important natural language downstream tasks, Part-of-speech tagging, Named-entity recognition, and Question answering. |
| Outcome: | The proposed model outperforms the existing model on three important natural language downstream tasks, Part-of-speech tagging, Named-entity recognition, and Question answering. |
Similar Papers
PhoBERT: Pre-trained language models for Vietnamese (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show that PhoBERT outperforms the recent best pre-trained multilingual model XLM-R in multiple Vietnamese-specific NLP tasks. |
| Approach: | They present PhoBERT with two versions, Phobert-base and PhoBRET-large, which are pre-trained for Vietnamese. |
| Outcome: | The proposed model outperforms the best pre-trained model XLM-R and improves the state-of-the-art in multiple Vietnamese-specific NLP tasks including Part-of speech tagging, Dependency parsing, Named-entity recognition and Natural language inference. |
ViHealthBERT: Pre-trained Language Models for Vietnamese in Health Text Mining (2022.lrec-1)
Copied to clipboard
| Challenge: | Recent large-scale language models show remarkable achievements in key NLP tasks such as Question Answering and Text Summarization. |
| Approach: | They propose a domain-specific pre-trained Vietnamese language model that outperforms the general domain language models. |
| Outcome: | The proposed model outperforms the general domain language models in Vietnamese datasets while outperforming the general-domain language models. |
ViT5: Pretrained Text-to-Text Transformer for Vietnamese Language Generation (2022.naacl-srw)
Copied to clipboard
| Challenge: | Existing models for the English language have been used to train on large corpus of high-quality texts. |
| Approach: | They present a pretrained Transformer-based encoder-decoder model for the Vietnamese language . they benchmark ViT5 on two downstream text generation tasks . |
| Outcome: | The proposed model outperforms existing models on Vietnamese Abstractive Summarization and Named Entity Recognition tasks. |
ViSoBERT: A Pre-Trained Language Model for Vietnamese Social Media Text Processing (2023.emnlp-main)
Copied to clipboard
| Challenge: | English and Chinese have seen the strong development of transformer-based language models for natural language processing tasks. |
| Approach: | They present a monolingual pre-trained language model for Vietnamese social media texts . they explore emotion recognition, hate speech detection, sentiment analysis, spam reviews detection . |
| Outcome: | The proposed model outperforms the existing models on Vietnamese social media tasks with fewer parameters. |
ViGLUE: A Vietnamese General Language Understanding Benchmark and Analysis of Vietnamese Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing benchmarks for natural language understanding have been suggested, but there is a lack of such a benchmark in Vietnamese due to the difficulty in accessing datasets or the scarcity of task-specific datasets. |
| Approach: | They propose to use a benchmark to evaluate Vietnamese language models in a variety of tasks and areas to explore the relationship between specific tasks and the number of shots. |
| Outcome: | The proposed benchmark contains twelve tasks and encompasses over ten areas and subjects, enabling it to evaluate models comprehensively over a broad spectrum of aspects. |
ViLegalLM: Language Models for Vietnamese Legal Text (2026.findings-acl)
Copied to clipboard
| Challenge: | **ViLegalLM** is the first suite of Vietnamese pretrained language models for legal text understanding and generation. |
| Approach: | They propose a suite of Vietnamese pretrained language models for legal text understanding and generation. |
| Outcome: | The proposed models outperform instruction-tuned adaptation on four main Vietnamese legal downstream tasks. |
VLUE: A New Benchmark and Multi-task Knowledge Transfer Learning for Vietnamese Natural Language Understanding (2024.findings-naacl)
Copied to clipboard
| Challenge: | a lack of standard evaluation metrics and benchmarks makes it difficult to identify strengths of Vietnamese NLP models. |
| Approach: | They propose to establish a standardized set of benchmarks for Vietnamese NLU . they propose to evaluate Vietnamese language understanding models using a pre-trained model . |
| Outcome: | The proposed model combines proficiency of a multilingual pre-trained model with Vietnamese linguistic knowledge. |
ViHateT5: Enhancing Hate Speech Detection in Vietnamese With a Unified Text-to-Text Transformer Model (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for HSD in Vietnamese focus on fine-tuning general pre-trained models, primarily trained on formal textual datasets like Wikipedia. |
| Approach: | They propose a T5-based HSD model pre-trained on a domain-specific dataset . their results highlight the significance of label distribution in pre-training data on model efficacy. |
| Outcome: | The proposed model can tackle multiple tasks using a unified model and achieve state-of-the-art performance across all standard HSD benchmarks in Vietnamese. |
FlauBERT: Unsupervised Language Model Pre-training for French (2020.lrec-1)
Copied to clipboard
Hang Le, Loïc Vial, Jibril Frej, Vincent Segonne, Maximin Coavoux, Benjamin Lecouteux, Alexandre Allauzen, Benoit Crabbé, Laurent Besacier, Didier Schwab
| Challenge: | Language models are a key step to achieve state-of-the-art results in many different Natural Language Processing (NLP) tasks. |
| Approach: | They propose to use a language model that is pre-trained on a large and heterogeneous French corpus to train continuous word representations. |
| Outcome: | The proposed model outperforms existing models on a large and heterogeneous French corpus. |
ViGPTQA - State-of-the-Art LLMs for Vietnamese Question Answering: System Overview, Core Models Training, and Evaluations (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Large language models (LLMs) and their applications in low-resource languages are limited due to lack of training data and benchmarking datasets. |
| Approach: | They propose a question-response system for Vietnamese that uses LLMs . they propose to open-source the model and train it on benchmark datasets based on Vietnamese data . |
| Outcome: | The proposed question answering system for Vietnamese is open-source and performant . it can learn and capture human-like text, but there is a gap in evaluations for Vietnamese . |