Papers by Nurul Aisyah
Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on large language models based on English datasets do not provide adequate data for evaluating their capabilities beyond English. |
| Approach: | They propose a multi-task language understanding benchmark for Indonesian culture and languages . it measures language proficiency, reasoning abilities and real-world knowledge . |
| Outcome: | The proposed model passes the primary school level in Indonesia, while other models perform at lower levels. |