| Challenge: | **ViLegalLM** is the first suite of Vietnamese pretrained language models for legal text understanding and generation. |
| Approach: | They propose a suite of Vietnamese pretrained language models for legal text understanding and generation. |
| Outcome: | The proposed models outperform instruction-tuned adaptation on four main Vietnamese legal downstream tasks. |
Similar Papers
ViDeBERTa: A powerful pre-trained language model for Vietnamese (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing models for Vietnamese that perform well on downstream tasks, such as Question answering, are based on Transformer. |
| Approach: | They propose a pre-trained monolingual Vietnamese model with three versions . they fine-tune and evaluate the model on three important natural language downstream tasks, Part-of-speech tagging, Named-entity recognition, and Question answering. |
| Outcome: | The proposed model outperforms the existing model on three important natural language downstream tasks, Part-of-speech tagging, Named-entity recognition, and Question answering. |
PhoBERT: Pre-trained language models for Vietnamese (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show that PhoBERT outperforms the recent best pre-trained multilingual model XLM-R in multiple Vietnamese-specific NLP tasks. |
| Approach: | They present PhoBERT with two versions, Phobert-base and PhoBRET-large, which are pre-trained for Vietnamese. |
| Outcome: | The proposed model outperforms the best pre-trained model XLM-R and improves the state-of-the-art in multiple Vietnamese-specific NLP tasks including Part-of speech tagging, Dependency parsing, Named-entity recognition and Natural language inference. |
ViLexNorm: A Lexical Normalization Corpus for Vietnamese Social Media Text (2024.eacl-long)
Copied to clipboard
| Challenge: | Lexical normalization is a fundamental task in Natural Language Processing (NLP) it involves the transformation of words into their canonical forms. |
| Approach: | They present a corpus of Vietnamese words annotated by human annotators for the Vietnamese lexical normalization task. |
| Outcome: | The best-performing system achieved 57.74% using the Error Reduction Rate (ERR) metric with the Leave-As-Is (LAI) baseline. |
ViT5: Pretrained Text-to-Text Transformer for Vietnamese Language Generation (2022.naacl-srw)
Copied to clipboard
| Challenge: | Existing models for the English language have been used to train on large corpus of high-quality texts. |
| Approach: | They present a pretrained Transformer-based encoder-decoder model for the Vietnamese language . they benchmark ViT5 on two downstream text generation tasks . |
| Outcome: | The proposed model outperforms existing models on Vietnamese Abstractive Summarization and Named Entity Recognition tasks. |
ViSoBERT: A Pre-Trained Language Model for Vietnamese Social Media Text Processing (2023.emnlp-main)
Copied to clipboard
| Challenge: | English and Chinese have seen the strong development of transformer-based language models for natural language processing tasks. |
| Approach: | They present a monolingual pre-trained language model for Vietnamese social media texts . they explore emotion recognition, hate speech detection, sentiment analysis, spam reviews detection . |
| Outcome: | The proposed model outperforms the existing models on Vietnamese social media tasks with fewer parameters. |
ViGPTQA - State-of-the-Art LLMs for Vietnamese Question Answering: System Overview, Core Models Training, and Evaluations (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Large language models (LLMs) and their applications in low-resource languages are limited due to lack of training data and benchmarking datasets. |
| Approach: | They propose a question-response system for Vietnamese that uses LLMs . they propose to open-source the model and train it on benchmark datasets based on Vietnamese data . |
| Outcome: | The proposed question answering system for Vietnamese is open-source and performant . it can learn and capture human-like text, but there is a gap in evaluations for Vietnamese . |
Vietnamese Automatic Speech Recognition: A Revisit (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing datasets with low quality and inconsistent annotations are insufficient for high-quality models. |
| Approach: | They propose a pipeline for aggregating and preprocessing high-quality ASR datasets from diverse, potentially noisy, open-source sources. |
| Outcome: | The proposed pipeline provides a foundation for training and evaluating state-of-the-art Vietnamese ASR systems. |
Improving Vietnamese-English Cross-Lingual Retrieval for Legal and General Domains (2025.naacl-short)
Copied to clipboard
Toan Ngoc Nguyen, Nam Le Hai, Nguyen Doan Hieu, Dai An Nguyen, Linh Ngo Van, Thien Huu Nguyen, Sang Dinh
| Challenge: | Existing document retrieval systems focus on a single language, targeting resource-rich languages like English or Chinese. |
| Approach: | They propose auxiliary loss function and symmetrical training strategy for cross-lingual retrieval between Vietnamese and English . they propose a dataset that covers the general domain and extends to the legal field . |
| Outcome: | The proposed dataset significantly improves state-of-the-art models on cross-lingual retrieval tasks. |
ViNLI: A Vietnamese Corpus for Studies on Open-Domain Natural Language Inference (2022.coling-1)
Copied to clipboard
| Challenge: | a large-scale corpus is needed for studies on natural language inference (NLI) for Vietnamese, which can be considered a low-resource language. |
| Approach: | They propose a corpus for evaluating Vietnamese natural language inference models . they use a human-annotated corpus extracted from more than 800 online news articles . |
| Outcome: | The ViNLI corpus is created and evaluated with a strict process of quality control . the best system performance is still far from human performance (a 14.20% gap in accuracy). |
Crossing Linguistic Horizons: Finetuning and Comprehensive Evaluation of Vietnamese Large Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing open-source LLMs exhibit limited effectiveness in processing Vietnamese . lack of systematic benchmark datasets and metrics tailored for Vietnamese LLM evaluation exacerbates these issues. |
| Approach: | They propose to fine tune LLMs specifically for Vietnamese and develop a framework for evaluation . they find that larger models introduce more biases and uncalibrated outputs . |
| Outcome: | The proposed framework finetunes LLMs specifically for Vietnamese and provides a framework for evaluation . |