Multilingual Native Language Identification with Large Language Models (2025.naacl-srw)
Copied to clipboard
| Challenge: | Native Language Identification (NLI) is the task of automatically identifying the native language (L1) of individuals based on their second language production. |
| Approach: | They evaluated the performance of several LLMs on non-English NLI corpora compared to traditional statistical machine learning models and language-specific BERT-based models. |
| Outcome: | The proposed models outperform statistical models and language-specific BERT-based models on English, Italian, Norwegian, and Portuguese. |
Similar Papers
A Deep Generative Approach to Native Language Identification (2020.coling-main)
Copied to clipboard
| Challenge: | Native language identification (NLI) is a multi-class classification task involving multiple features that capture the systematic fingerprints of the first language in the second language writing. |
| Approach: | They propose a deep generative language modelling approach to NLI that fine-tunes a GPT-2 model separately on texts written by the authors with the same L1 and assigns n-grams to an unseen text. |
| Outcome: | The proposed method outperforms traditional machine learning approaches and currently achieves the best results on the benchmark NLI datasets. |
Native Language Identification in Texts: A Survey (2024.naacl-long)
Copied to clipboard
| Challenge: | Native language identification is the task of automatically identifying an author’s native language (L1) based on their second language production. |
| Approach: | They present a survey of native language identification applied to texts . authors describe several text representations and computational techniques used in the task . |
| Outcome: | The proposed task has been widely studied for both text and speech, particularly for L2 English due to the availability of suitable corpora. |
BigNLI: Native Language Identification with Big Bird Embeddings (2024.lrec-main)
Copied to clipboard
| Challenge: | Native Language Identification (NLI) is a task that relies on time-consuming linguistic feature engineering and current transformer models are limited by input size. |
| Approach: | They propose to train a logistic regression classifier which only uses Big Bird embeddings to overcome this limitation. |
| Outcome: | The proposed method outperforms linguistic feature engineering models on the Reddit-L2 dataset and shows consistent out-of-sample and out-off-domain performance. |
Investigating Large Language Models for Complex Word Identification in Multilingual and Multidomain Setups (2024.emnlp-main)
Copied to clipboard
Răzvan-Alexandru Smădu, David-Gabriel Ion, Dumitru-Clementin Cercel, Florin Pop, Mihaela-Claudia Cercel
| Challenge: | Large language models (LLMs) are popular in the Natural Language Processing community because of their versatility and capability to solve unseen tasks in zero/few-shot settings. |
| Approach: | They investigate the use of large language models in CWI, LCP, and MWE settings by evaluating their use in zero-shot, few-shot and fine-tuning settings. |
| Outcome: | The proposed models struggle in certain conditions or achieve comparable results against existing methods. |
Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation? (2024.findings-eacl)
Copied to clipboard
Rishav Hada, Varun Gumma, Adrian Wynter, Harshita Diddee, Mohamed Ahmed, Monojit Choudhury, Kalika Bali, Sunayana Sitaram
| Challenge: | Large Language Models (LLMs) excel in various tasks, but their evaluation, especially in languages beyond the top 20, remains inadequate due to existing benchmarks and metrics limitations. |
| Approach: | They propose to use Large Language Models as evaluators to rank or score other models’ outputs by calibrating them against 20K human judgments across three text-generation tasks, five metrics, and eight languages. |
| Outcome: | The proposed evaluation methods can be used to improve multilingual evaluation by calibrating them against 20K human judgments across three text-generation tasks, five metrics, and eight languages. |
Leveraging Large Language Models for NLG Evaluation: Advances and Challenges (2024.emnlp-main)
Copied to clipboard
| Challenge: | introducing Large Language Models (LLMs) has opened new avenues for assessing generated content quality, e.g., coherence, creativity, and context relevance. |
| Approach: | They propose a taxonomy for organizing existing LLM-based evaluation metrics and a structured framework to understand and compare them. |
| Outcome: | The proposed taxonomy offers a framework to understand and compare LLM-based evaluation methods. |
ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in natural language processing (NLP) have led to significant breakthroughs in the field. |
| Approach: | They evaluate ChatGPT over multiple tasks with diverse languages and large datasets to provide more comprehensive information for multilingual NLP applications. |
| Outcome: | The proposed model can process and generate texts for multiple languages due to its multilingual training data. |
MEGA: Multilingual Evaluation of Generative AI (2023.emnlp-main)
Copied to clipboard
Kabir Ahuja, Harshita Diddee, Rishav Hada, Millicent Ochieng, Krithika Ramesh, Prachi Jain, Akshay Nambi, Tanuja Ganu, Sameer Segal, Mohamed Ahmed, Kalika Bali, Sunayana Sitaram
| Challenge: | Large Large Models (LLMs) have shown impressive performance on many natural language processing tasks such as language understanding, reasoning, and language generation. |
| Approach: | They present a framework for evaluating generative LLMs in the multilingual setting and provide directions for future progress in the field. |
| Outcome: | The proposed framework evaluates generative models on 16 NLP datasets across 70 typologically diverse languages and compares them to state-of-the-art non-autoregressive models. |
Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated multilingual capabilities, yet they are mostly English-centric due to the imbalanced training corpora. |
| Approach: | They extend the evaluation to real-world user queries and non-English-centric LLMs . they show that translation into English can boost LLM performance on NLP tasks . |
| Outcome: | The proposed evaluation extends to user queries and non-English-centric LLMs . it shows that translation into English can boost performance on NLP tasks, but not universally optimal . |
Identifying Open Challenges in Language Identification (2025.acl-long)
Copied to clipboard
| Challenge: | Existing work on language identification has focused on cross-domain setups, but no systematic comparison is available. |
| Approach: | They propose to train an accurate multi-domain languageidentification model on 2,034 languages and analyze the remaining errors. |
| Outcome: | The proposed model performs well on 2,034 languages with training with 1,000 instances per language and a maximum input length of 100 characters. |