IndoCL: Benchmarking Indonesian Language Development Assessment (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent interest has surged in applying natural language processing (NLP) and machine learning (ML) to evaluate language development in both first (L1) and second (L2) language acquisition. |
| Approach: | They propose to use an Indonesian corpus as a benchmark for LDA tasks and to use existing large-scale language models to improve performance. |
| Outcome: | The proposed model extracts language-independent features, relieving laborious computation and reliance on specific language. |
Similar Papers
IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding (2020.aacl-main)
Copied to clipboard
Bryan Wilie, Karissa Vincentio, Genta Indra Winata, Samuel Cahyawijaya, Xiaohong Li, Zhi Yuan Lim, Sidik Soleman, Rahmad Mahendra, Pascale Fung, Syafri Bahar, Ayu Purwarianti
| Challenge: | Despite the availability of data on Indonesian, progress on this language is slow . available datasets are scattered, with a lack of documentation and minimal community engagement. |
| Approach: | They propose a resource for training, evaluation, and benchmarking on Indonesian natural language understanding tasks. |
| Outcome: | The proposed resource includes 12 tasks ranging from single sentence classification to pair-sentences sequence labeling with different levels of complexity. |
IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation (2021.emnlp-main)
Copied to clipboard
Samuel Cahyawijaya, Genta Indra Winata, Bryan Wilie, Karissa Vincentio, Xiaohong Li, Adhiguna Kuncoro, Sebastian Ruder, Zhi Yuan Lim, Syafri Bahar, Masayu Khodra, Ayu Purwarianti, Pascale Fung
| Challenge: | Lack of publicly available NLG benchmarks for low-resource languages poses a challenge . authors show that IndoBART and IndoGPT achieve competitive performance on all tasks . |
| Approach: | They propose a benchmark to measure natural language generation progress in three low-resource languages of Indonesia . they use a corpus of pretraining datasets to build their models . |
| Outcome: | The proposed benchmark measures progress in Indonesian, Javanese, and Sundanese . the results highlight the importance of pretraining on closely related, localized languages . |
LORAXBENCH: A Multitask, Multilingual Benchmark Suite for 20 Indonesian Languages (2025.emnlp-main)
Copied to clipboard
| Challenge: | LORAXBENCH is a benchmark for low-resource languages of Indonesia . it covers reading comprehension, open domain QA, language inference, causal reasoning, translation, and cultural question answering across 20 languages. |
| Approach: | They propose a benchmark that focuses on low-resource languages of Indonesia and covers 6 diverse tasks: reading comprehension, open-domain QA, language inference, causal reasoning, translation, and cultural question answering. |
| Outcome: | The proposed benchmark covers reading comprehension, open-domain QA, language inference, causal reasoning, translation, and cultural question answering across 20 Indonesian languages. |
IndoLEM and IndoBERT: A Benchmark Dataset and Pre-trained Language Model for Indonesian NLP (2020.coling-main)
Copied to clipboard
| Challenge: | despite being spoken by 200 million people, the Indonesian language is underrepresented in NLP research. |
| Approach: | They propose a dataset for Indonesian that includes seven NLP tasks . they also propose 'indonesian language evaluation Montage' tasks that are based on previous work . |
| Outcome: | The proposed dataset shows that IndoBERT outperforms IndoLEM over most of the tasks. |
Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on large language models based on English datasets do not provide adequate data for evaluating their capabilities beyond English. |
| Approach: | They propose a multi-task language understanding benchmark for Indonesian culture and languages . it measures language proficiency, reasoning abilities and real-world knowledge . |
| Outcome: | The proposed model passes the primary school level in Indonesia, while other models perform at lower levels. |
Sinhala Encoder-only Language Models and Evaluation (2025.acl-long)
Copied to clipboard
Tharindu Ranasinghe, Hansi Hettiarachchi, Nadeesha Chathurangi Naradde Vidana Pathirana, Damith Premasiri, Lasitha Uyangodage, Isuri Nanomi Arachchige, Alistair Plum, Paul Rayson, Ruslan Mitkov
| Challenge: | Recent advances in language models (LMs) have produced excellent results in many NLP tasks, but their effectiveness is highly dependent on available pre-training resources. |
| Approach: | They propose to collect the largest monolingual corpus for Sinhala and compile a benchmark and evaluate LMs on it. |
| Outcome: | The proposed language models outperform the popular multilingual LMs in downstream NLP tasks. |
Bhaasha, Bhāṣā, Zaban: A Survey for Low-Resourced Languages in South Asia – Current Stage and Challenges (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a survey examines the current efforts and challenges of NLP models for South Asian languages . there are more than 650 languages in South Asia, but many have very limited computational resources or are missing from existing models. |
| Approach: | a survey examines efforts and challenges of NLP for South Asian languages . they focus on transformer-based models such as BERT, T5, & GPT . findings highlight substantial issues, including missing data in critical domains . |
| Outcome: | The findings highlight significant issues, including missing data in critical domains . the survey aims to raise awareness within the NLP community for more targeted data curation . |
FormosanBench: Benchmarking Low-Resource Austronesian Languages in the Era of Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing LLMs consistently underperform across all tasks, with 10-shot learning and fine-tuning offering only limited improvements. |
| Approach: | They introduce FormosanBench, a benchmark for evaluating LLMs on low-resource Austronesian languages. |
| Outcome: | The proposed benchmark covers three endangered Formosan languages: Atayal, Amis, and Paiwan . existing LLMs consistently underperform across all tasks, with 10-shot learning and fine-tuning offering only limited improvements. |
Scoping natural language processing in Indonesian and Malay for education applications (2022.acl-srw)
Copied to clipboard
| Challenge: | Limited natural language processing resources are available for Indonesian and Malay varieties and are difficult to locate. |
| Approach: | They propose to encourage collaboration and efficiency within NLP in Indonesian and Malay by identifying most published authors and research hubs. |
| Outcome: | The findings suggest that the field is dominated by exploratory corpus work, machine reading of text gathered from the Internet, and sentiment analysis. |
IndicNLPSuite: Monolingual Corpora, Evaluation Benchmarks and Pre-trained Multilingual Language Models for Indian Languages (2020.findings-emnlp)
Copied to clipboard
Divyanshu Kakwani, Anoop Kunchukuttan, Satish Golla, Gokul N.C., Avik Bhattacharyya, Mitesh M. Khapra, Pratyush Kumar
| Challenge: | In this paper, we present NLP resources for 11 major Indian languages . distributional representations are the cornerstone of modern NLP, authors say . |
| Approach: | They introduce NLP resources for 11 major Indian languages from two major language families . monolingual corpora contains 8.8 billion tokens across all 11 languages and Indian English . they also compile a benchmark for Indian language NLU to evaluate their results . |
| Outcome: | The monolingual corpora contains 8.8 billion tokens across all 11 languages and Indian English . the pre-trained language models are based on the compact ALBERT model . |