Papers with English
KreolMorisienMT: A Dataset for Mauritian Creole Machine Translation (2022.findings-aacl)
Copied to clipboard
| Challenge: | Mauritian Creole is a French-based creole and a lingua franca of the Republic of Mauritius. |
| Approach: | They describe a dataset for benchmarking machine translation quality of Mauritian Creole. |
| Outcome: | The proposed dataset compares KreolMorisienMT with existing models and human evaluation reveals the systems’ high translation quality. |
Enhancing Entity Boundary Detection for Better Chinese Named Entity Recognition (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to Chinese Named Entity Recognition (NER) lack explicit word boundary and tenses information. |
| Approach: | They propose a boundary enhanced approach for Chinese Named Entity Recognition . they add an additional Graph Attention Network(GAT) layer to capture internal dependency of phrases . |
| Outcome: | The proposed approach improves Chinese Named Entity Recognition (NER) on OntoNotes and Weibo corpora. |
Meta-Learning a Cross-lingual Manifold for Semantic Parsing (2023.tacl-1)
Copied to clipboard
| Challenge: | Recent work has found success with machine translation or zero-shot methods . however, these approaches can struggle to model how native speakers ask questions . |
| Approach: | They propose a meta-learning algorithm to leverage minimal annotated examples in new languages for few-shot cross-lingual semantic parsing. |
| Outcome: | The proposed approach trains a parser with maximum sample efficiency in six languages on ATIS. |
Aligning Multilingual Word Embeddings for Cross-Modal Retrieval Task (D19-66)
Copied to clipboard
| Challenge: | Existing methods to learn multimodal multilingual embeddings for text and image retrieval tasks are limited to English. |
| Approach: | They propose a new approach to learn multimodal multilingual embeddings for matching images and captions in two languages by combing two existing objective functions and adapting alignment between existing languages. |
| Outcome: | The proposed model achieves state-of-the-art in retrieval and caption-caption tasks while adapting existing language alignments. |
Injecting Wiktionary to improve token-level contextual representations using contrastive learning (2024.eacl-short)
Copied to clipboard
| Challenge: | lexical semantics tasks require contextual word embeddings that are not blind to context, despite the fact that vectors of the same meaning are too different. |
| Approach: | They propose to fine-tune pre-trained language models by using automatically self-augmented examples to target contextual word embeddings. |
| Outcome: | The proposed method achieves significant improvements on the original WiC test set and in two new tests. |
Detecting Sexism in Tweets: A Sentiment Analysis and Graph Neural Network Approach (2025.naacl-srw)
Copied to clipboard
Diana P. Madera-Espíndola, Zoe Caballero-Domínguez, Valeria J. Ramírez-Macías, Sabur Butt, Hector Ceballos
| Challenge: | a new tool to detect sexism on social media platforms is being developed to identify such behavior . sexist ideologies such as sextism and gender-based violence can be spread through social media . |
| Approach: | They propose to use BERT and GraphSAGE to analyze tweets for sexism detection . they also use sentiment analysis and natural language processing techniques to classify tweets . |
| Outcome: | The proposed tool analyzes tweets for sexism detection and classifies them into five categories. |
Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinement (2021.tacl-1)
Copied to clipboard
| Challenge: | RNGTr is a non-recursive Graph-to-Graph Transformer for iterative refinement of graphs . it can improve the accuracy of initial parsers on 13 languages . |
| Approach: | They propose a recursive non-autoregressive Graph-to-Graph Transformer architecture for iterative refinement of arbitrary graphs and apply it to syntactic dependency parsing. |
| Outcome: | The proposed architecture can improve state-of-the-art on 13 languages and the German CoNLL2009 corpus. |
InstructGEC: Enhancing Unsupervised Grammatical Error Correction with Instruction Tuning (2025.coling-main)
Copied to clipboard
| Challenge: | Recent studies have proposed methods of generating synthetic data for unsupervised GEC . however, the cost of such methods is high and the quality of the data is poor . |
| Approach: | They propose a method to generate synthetic data automatically for unsupervised GEC . they use a masking strategy to mask an erroneous sentence and the instruction consistently . |
| Outcome: | The proposed method outperforms state-of-the-art unsupervised methods on English and Chinese GEC datasets. |
The OSU/Facebook Realizer for SRST 2019: Seq2Seq Inflection and Serialized Tree2Tree Linearization (D19-63)
Copied to clipboard
| Challenge: | Existing linearization methods for shallow surface realization tasks are not available for all languages. |
| Approach: | They propose a system that implements morphological inflection with a baseline linearizer for a shallow surface realization task. |
| Outcome: | The proposed system is competitive across languages, but poor on longer sentences. |
Improving Vietnamese-English Cross-Lingual Retrieval for Legal and General Domains (2025.naacl-short)
Copied to clipboard
Toan Ngoc Nguyen, Nam Le Hai, Nguyen Doan Hieu, Dai An Nguyen, Linh Ngo Van, Thien Huu Nguyen, Sang Dinh
| Challenge: | Existing document retrieval systems focus on a single language, targeting resource-rich languages like English or Chinese. |
| Approach: | They propose auxiliary loss function and symmetrical training strategy for cross-lingual retrieval between Vietnamese and English . they propose a dataset that covers the general domain and extends to the legal field . |
| Outcome: | The proposed dataset significantly improves state-of-the-art models on cross-lingual retrieval tasks. |
RU-ADEPT: Russian Anonymized Dataset with Eight Personality Traits (2022.lrec-1)
Copied to clipboard
C. Anton Rytting, Valerie Novak, James R. Hull, Victor M. Frank, Paul Rodrigues, Jarrett G. W. Lee, Laurel Miller-Sims
| Challenge: | Social media has provided a platform for many individuals to express themselves naturally and publicly, but most of the work in this area has focused on English and other Western European languages. |
| Approach: | They propose to use a Russian dataset to combine author trait data with social media content to find out how personality traits are manifested. |
| Outcome: | The proposed dataset is the first to associate demographic and personality trait data with Russian-language social media content and to a limited extent, the first publicly-available dataset of personality traits to author content across multiple social media platforms. |
Coreference Resolution through a seq2seq Transition-Based System (2023.tacl-1)
Copied to clipboard
| Challenge: | Recent coreference resolution systems use search algorithms to identify mentions and resolve coreference. |
| Approach: | They propose a text-to-text coreference resolution system that uses a semantic paradigm to predict mentions and links jointly. |
| Outcome: | The proposed system achieves state-of-the-art accuracy on CoNLL-2012 datasets with 83.3 F1-score for English, 68.5 F1 score for Arabic, and 74.3 F1 scores for Chinese. |
QSpell 250K: A Large-Scale, Practical Dataset for Chinese Search Query Spell Correction (2025.naacl-industry)
Copied to clipboard
| Challenge: | Chinese Search Query Spell Correction is a task designed to identify and correct typographical errors within queries. |
| Approach: | They propose a large-scale benchmark specifically developed for Chinese Query Spell Correction. |
| Outcome: | The proposed benchmark covers a broad range of topics, including formal entities, everyday colloquialisms and idiomatic expressions. |
RankGen: Improving Text Generation with Large Ranking Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Modern language models assign high probabilities to output sequences that are repetitive, incoherent, or irrelevant to the prefix. |
| Approach: | They propose a 1.2B parameter encoder model for English that scores model generations given a prefix. |
| Outcome: | The proposed model outperforms decoding algorithms on automatic metrics and human evaluations with English writers. |
Constructing a Dataset for Hallucination Detection in Japanese Summarization with Fine-grained Faithfulness Labels (2026.eacl-srw)
Copied to clipboard
| Challenge: | Large language models (LLMs) can generate fluent text, but the quality of generated content depends on its consistency with the given input. |
| Approach: | They constructed a Japanese evaluation dataset for hallucination detection in summarization by manually annotating sentence-level faithfulness labels in LLM-generated summaries of Japanese documents. |
| Outcome: | The proposed model can detect hallucinations in Japanese documents by annotating faithfulness labels in Japanese summaries. |
VLUE: A New Benchmark and Multi-task Knowledge Transfer Learning for Vietnamese Natural Language Understanding (2024.findings-naacl)
Copied to clipboard
| Challenge: | a lack of standard evaluation metrics and benchmarks makes it difficult to identify strengths of Vietnamese NLP models. |
| Approach: | They propose to establish a standardized set of benchmarks for Vietnamese NLU . they propose to evaluate Vietnamese language understanding models using a pre-trained model . |
| Outcome: | The proposed model combines proficiency of a multilingual pre-trained model with Vietnamese linguistic knowledge. |
GPTAraEval: A Comprehensive Evaluation of ChatGPT on Arabic NLP (2023.emnlp-main)
Copied to clipboard
| Challenge: | Our study examines ChatGPT’s performance on Arabic languages and dialectal varieties. |
| Approach: | They conduct a large-scale automated and human evaluation of ChatGPT, encompassing 44 distinct language understanding and generation tasks on over 60 different datasets. |
| Outcome: | The proposed model outperforms smaller models on Arabic dialects compared to GPT-4's Modern Standard Arabic and Dialectal Arabic (DA) |
Language Technologies for the Creation of Multilingual Terminologies. Lessons Learned from the SSHOC Project (2022.lrec-1)
Copied to clipboard
| Challenge: | Language Technologies can help in promoting and facilitating multilingualism in the Social Sciences and Humanities domain. |
| Approach: | They propose to use Natural Language Processing and Machine Translation to provide tools to foster multilingual access and discovery to SSH content across different languages. |
| Outcome: | The proposed tools prove to be a valid asset to translation tasks . validation of results by domain experts proficient in the language is an unavoidable phase of the whole workflow. |
FastFit: Fast and Effective Few-Shot Text Classification with a Multitude of Classes (2024.naacl-demo)
Copied to clipboard
| Challenge: | Few-shot prompting of large language models (LLMs) via API calls presents a unique challenge when dealing with a multitude of classes that share similar semantic meanings. |
| Approach: | They present a Python package that integrates batch contrastive learning and token-level similarity score to provide fast few-shot classification. |
| Outcome: | The proposed method significantly improves multi-class classification speed and accuracy across English and Multilingual datasets. |
Representing ELMo embeddings as two-dimensional text online (2021.eacl-demos)
Copied to clipboard
| Challenge: | ELMoViz module adds support for contextualized embedding architectures, in particular for token embeddable word models. |
| Approach: | They propose to add a module to the free and open-source WebVectors toolkit which provides lexical hyperlinks to word representations in static embedding models. |
| Outcome: | The ELMoViz module adds support for contextualized embedding architectures, in particular for ELMa models. |
A Computational Analysis of the Dehumanisation of Migrants from Syria and Ukraine in Slovene News Media (2024.lrec-main)
Copied to clipboard
| Challenge: | Dehumanisation involves the perception and/or treatment of a social group’s members as less than human. |
| Approach: | They propose to use a new sentiment resource to make it easier to transfer to other languages and to evaluate and use . they then apply the method to study attitudes to migration expressed in Slovene newspapers, and examine how this discourse changed between the 2015-16 migration crisis and the 2022-23 period following the war in Ukraine. |
| Outcome: | The proposed method is easier to transfer to other languages and evaluates . it combines zero-shot cross-lingual valence and arousal detection with statistical significance testing to examine attitudes to migration expressed in Slovene newspapers . |
PEEP-Talk: A Situational Dialogue-based Chatbot for English Education (2023.acl-demo)
Copied to clipboard
Seungjun Lee, Yoonna Jang, Chanjun Park, Jungseob Lee, Jaehyung Seo, Hyeonseok Moon, Sugyeong Eo, Seounghoon Lee, Bernardo Yahya, Heuiseok Lim
| Challenge: | Existing chatbots lack realistic practice scenarios for English learners . existing platforms employ hand-crafted and patternmatching rules, limiting communication ability and responding appropriately to out-of-situation utterances. |
| Approach: | They propose a real-world situational dialogue-based chatbot for English education . it generates appropriate responses in various real-life situations while providing accurate feedback . |
| Outcome: | The proposed chatbot generates appropriate responses in various real-life situations while providing accurate feedback to learners. |
mOKB6: A Multilingual Open Knowledge Base Completion Benchmark (2023.acl-short)
Copied to clipboard
| Challenge: | Open knowledge bases (Open KBCs) are constructed from triples of the form, which can be denoted as (s, r, o) by using open information extraction (Open IE) systems. |
| Approach: | They construct a dataset with facts from Wikipedia in six languages . they use open information extraction systems to extract triples from text . |
| Outcome: | The proposed dataset contains facts from Wikipedia in six languages including English . it improves existing Open KB construction pipeline by doing multilingual coreference resolution and keeping only entity-linked triples . |
Multilingual BERT Post-Pretraining Alignment (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work improves on the success of monolingual pretrained language models by adding cross-lingual tasks that always involve English. |
| Approach: | They propose a method to align multilingual contextual embeddings as a post-pretraining step for improved cross-lingual transferability of pretrained language models. |
| Outcome: | The proposed model outperforms XLM-R_Base on translation-train tasks while using less parallel data and fewer parameters. |
Tab-CQA: A Tabular Conversational Question Answering Dataset on Financial Reports (2023.acl-industry)
Copied to clipboard
| Challenge: | Existing conversational question answering datasets are usually constructed from unstructured texts in English. |
| Approach: | They propose a Chinese tabular conversational question answering dataset based on financial reports . they select 2,463 tables and manually generate 2,463, conversations with 35,494 QA pairs . |
| Outcome: | The proposed dataset is based on Chinese financial reports extracted from listed companies in the past 30 years. |
Knowledge extraction from aeronautical messages (NOTAMs) with self-supervised language models for aircraft pilots (2022.naacl-industry)
Copied to clipboard
| Challenge: | During pre-flight briefings, aircraft pilots analyse a long list of NOTAMs . the messages are usually written in the English language, but the phrasing is very special . |
| Approach: | They pretrain language models derived from BERT on circa 1 million unlabeled NOTAMs . they reuse the learnt representations on three downstream tasks valuable for pilots - criticality prediction, named entity recognition and translation into a structured language called Airlang. |
| Outcome: | The proposed language model can be used on criticality prediction, named entity recognition and translation into a structured language called Airlang. |
Efficient Neural Machine Translation for Low-Resource Languages via Exploiting Related Languages (2020.acl-srw)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) is a rapidly advancing MT paradigm that can be used to improve machine translation for many languages. |
| Approach: | They propose a technique called Unified Transliteration and Subword Segmentation to leverage language similarity while exploiting parallel data from related languages. |
| Outcome: | The proposed approach improves translation accuracy by 5 BLEU points over the standard Transformer-based NMT models. |
Does Character-level Information Always Improve DRS-based Semantic Parsing? (2023.starsem-1)
Copied to clipboard
| Challenge: | incorporating character-level information does not improve the performance in English and German, and is not sensitive to correct character order in Dutch. |
| Approach: | They propose to incorporate character-level representations into a neural semantic parser for Discourse Representation Structures and to test their performance using order of character sequences. |
| Outcome: | The proposed parser improves in English, German, Dutch, and Italian in four languages. |
Pun2Pun: Benchmarking LLMs on Textual-Visual Chinese-English Pun Translation via Pragmatics Model and Linguistic Reasoning (2025.acl-srw)
Copied to clipboard
| Challenge: | Current approaches resort to suboptimal compromises and computational methods remain inadequate for translation. |
| Approach: | They propose a Constant-Variable Optimization (CVO) model for translation strategy and an Ovl metric for translation quality assessment that adapts to Chinese and English. |
| Outcome: | The proposed model improves performance on textual and visual puns while maintaining linguistic mechanisms and humorous effects. |
Lattice Path Edit Distance: A Romanization-aware Edit Distance for Extracting Misspelling-Correction Pairs from Japanese Search Query Logs (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods to extract misspelling-correction pairs from Japanese query logs are not effective due to the unique input methods. |
| Approach: | They propose a romanization-aware edit distance that utilizes romanization lattices to efficiently consider all possible romanized forms of input strings. |
| Outcome: | Empirical results show lattice path edit distance outperforms standard edit distance in Japanese . latticae path editing distance outpersforms existing methods even with romanization . |
SAFARI: A Community-Engaged Approach and Dataset of Stereotype Resources in the Sub-Saharan African Context (2026.eacl-short)
Copied to clipboard
Aishwarya Verma, Laud Ammah, Olivia Nercy Ndlovu Lucas, Andrew Zaldivar, Vinodkumar Prabhakaran, Sunipa Dev
| Challenge: | Existing data collection approaches to generative AI are inadequate to assess its safety and utility. |
| Approach: | They propose a multilingual stereotype resource that uses socioculturally-situated, community-engaged methods to assess the region’s linguistic diversity and traditional orality. |
| Outcome: | The proposed method covers four sub-Saharan African countries that are severely underrepresented in NLP resources: Ghana, Kenya, Nigeria, and South Africa. |
Langsmith: An Interactive Academic Text Revision System (2020.emnlp-demos)
Copied to clipboard
| Challenge: | Currently, diversity and inclusion initiatives in the academic community are encouraged . however, writing papers in English can be a daunting task . |
| Approach: | They propose a system that helps non-native English speakers to write papers in English . the system can suggest fluent, academic-style sentences based on their rough, incomplete phrases or sentences . |
| Outcome: | The proposed system can help non-native English speakers write papers in English . the system can suggest fluent, academic-style sentences based on their rough sentences . |
X-Instruction: Aligning Language Model in Low-resource Languages with Self-curated Cross-lingual Instructions (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models respond well in high-resource languages but struggle in low-resourced languages. |
| Approach: | They propose a method to construct cross-lingual instruction following samples with instruction in English and response in low-resource languages. |
| Outcome: | The proposed method builds a large-scale cross-lingual instruction tuning dataset on 10 languages. |
Personalized Microblog Sentiment Classification via Adversarial Cross-lingual Multi-task Learning (D18-1)
Copied to clipboard
| Challenge: | Existing personalized microblog sentiment classification methods suffer from the insufficiency of discriminative tweets for personalization learning. |
| Approach: | They propose to use user-attention-based Convolutional Neural Networks to capture individuality and opinion bias in microblog posts and a novel adversarial cross-lingual learning framework to enrich the user post representation. |
| Outcome: | The proposed method outperforms state-of-the-art baseline algorithms with large margins on English and Chinese microblog datasets. |
SwissADT: An Audio Description Translation System for Swiss Languages (2025.naacl-industry)
Copied to clipboard
| Challenge: | despite advances in multilingual machine translation, lack of well-crafted AD data impedes development of audio description translation systems. |
| Approach: | They propose an audio description translation system for three main Swiss languages and English . they combine human expertise with the power of Large Language Models to improve quality . |
| Outcome: | The proposed system is designed to enhance accessibility for multilingual populations in Switzerland. |
Stop Pre-Training: Adapt Visual-Language Models to Unseen Languages (2023.acl-short)
Copied to clipboard
| Challenge: | Existing studies have shown that the pre-training in English does not transfer well to other languages in a zero-shot setting. |
| Approach: | They propose a simple yet efficient approach to adapt VLP to unseen languages using MPLM. |
| Outcome: | The proposed approach outperforms state-of-the-art models without large parallel corpora across three tasks. |
Basreh or Basra? Geoparsing Historical Locations in the Svoboda Diaries (2024.acl-srw)
Copied to clipboard
| Challenge: | In the historical domain, many geoparsing corpora are from large news collections. |
| Approach: | They propose a pipeline employing named entity recognition for geotagging and a map-based generate-and-rank approach incorporating candidate name augmentation and clustering of location context words for geocoding. |
| Outcome: | The proposed pipeline outperforms existing map-based geoparsers in terms of accuracy, lowest mean distance error, and number of locations correctly identified. |
StRuCom: A Novel Dataset of Structured Code Comments in Russian (2025.acl-srw)
Copied to clipboard
| Challenge: | Existing machine learning models for code comment generation are poorly suited for Russian . existing datasets that contain simple comments and docstrings in English are not suitable for function-level documentation generation. |
| Approach: | They propose a dataset specifically designed for Russian code documentation. |
| Outcome: | The first large-scale dataset specifically designed for Russian code documentation is based on human-written comments from GitHub repositories with synthetically generated ones. |
BanglaSTEM: A Parallel Corpus and Term-Weighted Evaluation for Technical Bangla-English Translation (2026.acl-srw)
Copied to clipboard
| Challenge: | Large language models excel at technical problem solving in English but struggle when questions are posed in Bangla. |
| Approach: | They propose a dataset of 5,000 Bangla-English sentence pairs to align technical terms . they use OCR to extract matching passages from bilingual textbooks . |
| Outcome: | The proposed pipeline extracts matching passages from bilingual textbooks and uses them to align sentences and mark technical terms. |
Multilingual Retrieval-Augmented Generation for Knowledge-Intensive Question Answering Task (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies focus on English as the data language for RAG, resulting in limited coverage of multilingual RAG. |
| Approach: | They propose a method that translates retrieved documents into a common language before generating the response. |
| Outcome: | The proposed approach improves efficiency on knowledge-intensive tasks but introduces inconsistencies due to cross-lingual variations in the retrieved content. |
How Much Do Language Models Copy From Their Training Data? Evaluating Linguistic Novelty in Text Generation Using RAVEN (2023.tacl-1)
Copied to clipboard
| Challenge: | Current language models generate high-quality text, but are they copying it or have they learned generalizable linguistic abstractions? |
| Approach: | They propose a suite of analyses for assessing the novelty of generated text . they focus on sequential structure (n-grams) and syntactic structure (syntactical structure). |
| Outcome: | The proposed model-generated text is as novel as the baseline human-generated model- generated text, but it is copied substantially, the authors show . |
Sluice Resolution without Hand-Crafted Features over Brittle Syntax Trees (N18-2)
Copied to clipboard
| Challenge: | sluice resolution in english is the problem of finding antecedents of wh-fronted ellipses . previous work relied on hand-crafted features over syntax trees that scale poorly to other languages and domains . |
| Approach: | They propose a model that uses partial parsing to find antecedents of wh-fronted ellipses in english . their model significantly outperforms previous work on available newswires . |
| Outcome: | The proposed model outperforms the only previous work on available newswires. |
KorNLI and KorSTS: New Benchmark Datasets for Korean Natural Language Understanding (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmark datasets for natural language inference and semantic textual similarity (STS) are not available in the Korean language. |
| Approach: | They construct and release new datasets for Korean NLI and STS . they machine-translate existing English training sets and manually translate development and test sets into Korean to accelerate research on Korean NLU. |
| Outcome: | The proposed datasets are available at https://github.com/kakaobrain/KorNLUDatasets. |
GOLEMcoref: A Multilingual Coreference Dataset of Fiction (2026.acl-short)
Copied to clipboard
Andreas Van Cranenburgh, Xiaoyan Yang, null Alvanita, Cecilia Nicole Di Domenico, Maria Ferragud, Arianna Graciotti, Byungjun Kim, Seonyeong Park, Noa Visser Solissa, Xiaoyu Zhou, Federico Pianzola
| Challenge: | Despite considerable progress, most research still focuses predominantly on English . fictional texts bring additional challenges not covered by standard benchmark datasets . |
| Approach: | They present a multilingual coreference dataset of 827k fanfiction tokens in 7 languages . they discuss their annotation scheme and language-specific challenges . |
| Outcome: | The proposed dataset includes full stories of diverse lengths, ranging from 500 to 17k words. |
Grammatical Error Correction in Low-Resource Scenarios (D19-55)
Copied to clipboard
| Challenge: | Existing systems for grammatical error correction in English have been limited . however, there is limited progress on error correction of other languages . |
| Approach: | They propose a dataset on grammatical error correction for Czech and an annotated learner corpus for Russian and Czech. |
| Outcome: | The proposed model can reach new state-of-the-art on Czech, German and Russian datasets. |
OmniEvent: A Comprehensive, Fair, and Easy-to-Use Toolkit for Event Understanding (2023.emnlp-demo)
Copied to clipboard
| Challenge: | Event understanding is fundamental for humans to understand the world. |
| Approach: | They propose an event understanding toolkit called OmniEvent that is comprehensive and fair . it supports mainstream modeling paradigms and the processing of 15 widely-used datasets . |
| Outcome: | The toolkit supports mainstream modeling paradigms and the processing of 15 widely-used English and Chinese datasets. |
Do Multilingual Language Models Think Better in English? (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing studies show that translation-test improves performance of multilingual models by translating the input into English using an external machine translation system. |
| Approach: | They propose a new approach that leverages the few-shot translation capabilities of multilingual language models. |
| Outcome: | The proposed approach outperforms direct inference on 5 tasks. |
SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval (2021.naacl-main)
Copied to clipboard
| Challenge: | SPARTA is a novel neural retrieval method for open-domain question answering . it learns a sparse representation that can be efficiently implemented as an Inverted Index . |
| Approach: | They propose a method that learns a sparse representation that can be implemented as an Inverted Index. |
| Outcome: | The proposed method achieves state-of-the-art results on 4 open-domain question answering tasks and 11 retrieval question answering (ReQA) tasks. |
IndicJR: A Judge-Free Benchmark of Jailbreak Robustness in South Asian Languages (2026.eacl-industry)
Copied to clipboard
| Challenge: | Indic Jailbreak Robustness (IJR) is a judge-free benchmark for adversarial safety across 12 languages. |
| Approach: | They propose a judge-free benchmark for adversarial safety across 12 languages . they find contracts inflate refusals but do not stop jailbreaks . |
| Outcome: | The proposed benchmarks cover 45,216 prompts in JSON and Free tracks. |
MTNT: A Testbed for Machine Translation of Noisy Text (D18-1)
Copied to clipboard
| Challenge: | Noisy input text can cause disastrous mistranslations in most modern machine translation systems. |
| Approach: | They propose a benchmark dataset for Machine Translation of Noisy Text (MTNT) they use reddit comments and professionally sourced translations to examine noise types. |
| Outcome: | The proposed dataset can provide an attractive testbed for noise-robust machine translation systems. |
Named Entity Recognition for Chinese biomedical patents (2020.coling-main)
Copied to clipboard
| Challenge: | Existing attempts to address NER for Chinese biomedical texts have been limited due to the amount of Chinese biomedicine discoveries being patented. |
| Approach: | They train and evaluate Chinese biomedical patents NER models based on BERT . their model is optimized for Chinese bio-patent data and scored an F1 . |
| Outcome: | The proposed model achieves an F1 score of 0.540.15 for Chinese biomedical patent data. |
Compositional Generalization in Multilingual Semantic Parsing over Wikidata (2022.tacl-1)
Copied to clipboard
| Challenge: | Semantic parsers are mostly designed for and evaluated on English resources, such as CFQ. |
| Approach: | They propose a method for creating a multilingual, parallel question-query dataset . they analyze compositional generalization of parsers in Hebrew, Kannada, Chinese, and English . |
| Outcome: | The proposed method analyzes compositional generalization of parsers in Hebrew, Kannada, Chinese, and English. |
UltraEval-Audio: A Unified Framework for Comprehensive Evaluation of Audio Foundation Models (2026.acl-demo)
Copied to clipboard
Qundong Shi, Jie Zhou, Biyuan Lin, Junbo Cui, Guoyang Zeng, Yixuan Zhou, Ziyang Wang, Xin Liu, Zhen Luo, Yudong Wang, Zhiyuan Liu
| Challenge: | Existing evaluation frameworks for audio foundation models are heavily reliant on English, making it difficult to objectively assess models’ performance on Chinese. |
| Approach: | They propose a unified framework that supports 10 languages, 14 task categories, 24 models, and 36 benchmarks with one-command evaluation and real-time leaderboards. |
| Outcome: | The proposed framework supports 10 languages, 14 task categories, 24 models, and 36 benchmarks with one-command evaluation and real-time leaderboards. |
Revisiting and Advancing Chinese Natural Language Understanding with Accelerated Heterogeneous Knowledge Pre-training (2022.emnlp-industry)
Copied to clipboard
Taolin Zhang, Junwei Dong, Jianing Wang, Chengyu Wang, Ang Wang, Yinghui Liu, Jun Huang, Yong Li, Xiaofeng He
| Challenge: | Existing knowledge-enhanced pre-trained language models (KEPLMs) can capture internal knowledge, but can't understand external background knowledge. |
| Approach: | They propose to use Chinese knowledge-enhanced pre-trained language models to improve context-aware representations via learning from structured relations in knowledge bases. |
| Outcome: | Experiments show that Chinese knowledge-enhanced pre-trained language models outperform strong baselines over various benchmark NLP tasks and in different model sizes. |
NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languages (2023.eacl-main)
Copied to clipboard
Genta Indra Winata, Alham Fikri Aji, Samuel Cahyawijaya, Rahmad Mahendra, Fajri Koto, Ade Romadhony, Kemal Kurniawan, David Moeljadi, Radityo Eko Prasojo, Pascale Fung, Timothy Baldwin, Jey Han Lau, Rico Sennrich, Sebastian Ruder
| Challenge: | In Indonesia, many languages are endangered and some are even extinct due to the unavailability of data resources and benchmarks. |
| Approach: | They propose a high-quality multilingual parallel corpus that covers 10 local languages from Indonesia. |
| Outcome: | The proposed resource includes sentiment and machine translation datasets, and bilingual lexicons. |
An Empirical Study on Fine-Grained Named Entity Recognition (C18-1)
Copied to clipboard
Khai Mai, Thai-Hoang Pham, Minh Trung Nguyen, Tuan Duc Nguyen, Danushka Bollegala, Ryohei Sasano, Satoshi Sekine
| Challenge: | Named entity recognition (NER) is a well studied topic in natural language processing. |
| Approach: | They propose to remove the CNN layer and use dictionary and category embeddings to improve Japanese FG-NER performance. |
| Outcome: | The proposed method improves Japanese FG-NER F-score from 66.76% to 75.18%. |
TimeRes: A Turkish Benchmark For Evaluating Temporal Understanding of Large Language Models (2026.eacl-srw)
Copied to clipboard
| Challenge: | Existing benchmarks focus on English and underexplore how linguistic structure contributes to temporal meaning. |
| Approach: | They propose a Turkish benchmark to evaluate temporal understanding of Large Language Models (LLMs) their benchmark examines Reichenbach’s temporal points and reported speech through date arithmetic . |
| Outcome: | The proposed model fails to resolve reported speech and fails to generalize across word order variations. |
Potential Idiomatic Expression (PIE)-English: Corpus for Classes of Idioms (2022.lrec-1)
Copied to clipboard
Tosin Adewumi, Roshanak Vadoodi, Aparajita Tripathy, Konstantina Nikolaido, Foteini Liwicki, Marcus Liwicki
| Challenge: | Potential Idiomatic Expression (PIE) dataset for NLP in English contains over 20,100 samples with almost 1,200 cases of idioms from 10 classes (or senses). |
| Approach: | They present a large Potential Idiomatic Expression (PIE) dataset for Natural Language Processing (NLP) in English. |
| Outcome: | The proposed dataset contains over 20,100 samples with almost 1,200 cases of idioms (with their meanings) from 10 classes (or senses). |
Beyond English: The Impact of Prompt Translation Strategies across Languages and Tasks in Multilingual LLMs (2025.findings-naacl)
Copied to clipboard
| Challenge: | Current LLMs are primarily trained on English data but also include data from other languages. |
| Approach: | They propose to use a pre-translation strategy to translate a task prompt into English before inference . they use 'a modular entity' that could be translated into four different languages . |
| Outcome: | The proposed strategies are based on a set of pre-trained data across 35 languages covering both low and high-resource languages. |
Adjusting the Precision-Recall Trade-Off with Align-and-Predict Decoding for Grammatical Error Correction (2022.acl-short)
Copied to clipboard
| Challenge: | Modern writing assistance applications always contain a Grammatical Error Correction (GEC) model to correct errors in user-entered sentences. |
| Approach: | They propose a simple yet effective approach to Align-and-Predict Decoding for most popular sequence-to-sequence models to offer more flexibility for the precision-recall trade-off. |
| Outcome: | The proposed model can be used in both English and Chinese GEC models and achieve state-of-the-art results. |
The EuroPat Corpus: A Parallel Corpus of European Patent Data (2022.lrec-1)
Copied to clipboard
| Challenge: | a new corpus of patent-specific parallel data is available for 6 official European languages paired with English: German, Spanish, French, Croatian, Norwegian, and Polish. |
| Approach: | They present a patent-specific corpus of parallel data for 6 official European languages paired with English: German, Spanish, French, Croatian, Norwegian, and Polish. |
| Outcome: | The filtered corpus ranges in size from 51 million sentences (Spanish-English) to 154k sentences (Croatian-English), with the unfiltered (raw) corpus being up to 2 times larger. |
MultiMed: Multilingual Medical Speech Recognition via Attention Encoder Decoder (2025.acl-industry)
Copied to clipboard
Khai Le-Duc, Phuc Phan, Tan-Hanh Pham, Bach Phan Tat, Minh-Huong Ngo, Thanh Nguyen-Tang, Truong-Son Hy
| Challenge: | Multilingual automatic speech recognition (ASR) in the medical domain is a critical foundational task, serving a wide range of downstream applications such as speech translation, spoken language understanding, and voice-activated assistants. |
| Approach: | They present the first multilingual medical ASR dataset and the first collection of small-to-large end-to end medical APR models spanning five languages: Vietnamese, English, German, French, and Mandarin Chinese. |
| Outcome: | The proposed model covers Vietnamese, English, German, French, and Mandarin Chinese, and is the first multilingual ASR dataset across five languages. |
Tonal Salience in Cognitive Decline: In-Context MCI Detection with Multimodal LLMs (2026.acl-srw)
Copied to clipboard
| Challenge: | a recent study shows that tonal languages like Chinese have a higher classification performance than non-tonal languages like English. |
| Approach: | a new study examines the differences between tonal and non-tonal language classifications . they hypothesize that the difference is rooted in language typology . early cognitive decline is notoriously difficult to detect . |
| Outcome: | The proposed method compared to TAUKADIAL audio shows that Chinese and English perform better on Chinese . the findings suggest that language typology should inform the design of audio-based cognitive screening tools . |
MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing multilingual QA datasets lack linguistic diversity and comparable evaluation between languages. |
| Approach: | They propose a multilingual question-answer evaluation set with 10k English queries and human translations of them into 25 additional languages and dialects. |
| Outcome: | The proposed model is based on a multilingual knowledge questions and answers evaluation set with 26 languages. |
Building Large-Scale English and Korean Datasets for Aspect-Level Sentiment Analysis in Automotive Domain (2020.coling-main)
Copied to clipboard
| Challenge: | Existing datasets in automotive domain cover only three languages due to high cost of human annotation. |
| Approach: | They build large-scale datasets of users’ comments in two languages, English and Korean, for aspect-level sentiment analysis in automotive domain. |
| Outcome: | The datasets consist of 58,000+ commentaspect pairs, which are the largest compared to existing datasets. |
Efficiently Aligned Cross-Lingual Transfer Learning for Conversational Tasks using Prompt-Tuning (2024.findings-eacl)
Copied to clipboard
| Challenge: | Cross-lingual transfer of language models trained on high-resource languages such as English has been limited due to the high cost of obtaining non-English conversational data. |
| Approach: | They introduce a parallel and large-scale multilingual conversation dataset that is used for cross-lingual alignment pretraining by translating the English-only Schema-Guided Dialogue dataset into 105 other languages. |
| Outcome: | The proposed model performs well on slot-filling and intent classification tasks, and is able to perform well in other languages. |
Beyond Accuracy: Alignment and Error Detection across Languages in the Bi-GSM8K Math-Teaching Benchmark (2026.findings-eacl)
Copied to clipboard
| Challenge: | Recent advances in LLMs have significantly improved mathematical problem-solving, with models like GPT-4 achieving human-level performance. |
| Approach: | They propose a bilingual English-Korean dataset enriched with teacher solutions, student solutions, and annotations marking students’ initial errors. |
| Outcome: | The proposed model achieves high agreement with human judgments and lower latency and resource usage than commercial APIs, demonstrating strong computational efficiency. |
NollySenti: Leveraging Transfer Learning and Machine Translation for Nigerian Movie Sentiment Classification (2023.acl-short)
Copied to clipboard
| Challenge: | Africa has over 2000 indigenous languages but they are under-represented in NLP research due to lack of datasets. |
| Approach: | They propose to use a dataset to classify sentiments for cross-domain adaptation for Nigerian and other African languages. |
| Outcome: | The proposed dataset compares the performance of cross-domain adaptation from Twitter domain and cross-lingual adaptation from English domain. |
Does Generative AI speak Nigerian-Pidgin?: Issues about Representativeness and Bias for Multilingualism in LLMs (2025.findings-naacl)
Copied to clipboard
| Challenge: | Nigeria is a multilingual country with 500+ languages. |
| Approach: | They propose to use a pidgin and a creole to analyze the pidgins of Nigeria . they also use machine translation to analyze their results . |
| Outcome: | The results show that the two pidgins do not represent each other and are hard to teach . the results show the pidgin varieties are underrepresented in Generative AI . |
Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to improve machine reading comprehension performance on low resource languages are limited due to the lack of sufficient training data. |
| Approach: | They propose to use a mixed MRC task to translate the question to other languages and build cross-lingual question-passage pairs. |
| Outcome: | The proposed task improves on two cross-lingual MRC datasets. |
AC-EVAL: Evaluating Ancient Chinese Language Understanding in Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | AC-EVAL is a benchmark designed to assess the advanced knowledge and reasoning capabilities of LLMs within the context of ancient Chinese. |
| Approach: | They propose a benchmark to assess the advanced knowledge and reasoning capabilities of LLMs in ancient Chinese. |
| Outcome: | AC-EVAL aims to assess the comprehension of ancient Chinese texts . the benchmark covers 13 tasks covering historical facts, geography, social customs, art, philosophy, classical poetry and prose. |
Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrieval (2022.coling-1)
Copied to clipboard
| Challenge: | State-of-the-art neural rankers are notoriously data-hungry and rarely used in multilingual and cross-lingual retrieval settings. |
| Approach: | They propose to use Sparse Fine-Tuning Masks and Adapters to transfer rankers trained on English data to other languages and cross-lingual setups by means of multilingual encoders. |
| Outcome: | The proposed methods outperform standard zero-shot transfer with full MMT fine-tuning while being more modular and reducing training times. |
Quantifying Synthesis and Fusion and their Impact on Machine Translation (2022.naacl-main)
Copied to clipboard
| Challenge: | Literature in Natural Language Processing (NLP) typically labels whole language with strict type of morphology, e.g. fusional or agglutinative. |
| Approach: | They propose to quantify morphological typology at the word and segment level by using two indices: synthesis (e.g. analytic to polysynthetic) and fusion (agglutinative to fusional). |
| Outcome: | The proposed method reduces the rigidity of NLP classification claims by measuring morphological diversity at the word and segment level. |
Low-Resource Sequence Labeling via Unsupervised Multilingual Contextualized Representations (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual sequence labeling require bilingual resources and require linguistic knowledge. |
| Approach: | They propose a multilingual language model with deep semantic Alignment to generate language-independent representations for cross-lingual sequence labeling. |
| Outcome: | The proposed model achieves state-of-the-art NER and POS performance across European languages and on distant language pairs such as English and Chinese. |
Detection-Correction Structure via General Language Model for Grammatical Error Correction (2024.acl-long)
Copied to clipboard
| Challenge: | Grammatical error correction (GEC) is a task dedicated to rectifying texts with minimal edits. |
| Approach: | They propose a detection-correction structure based on the general language model which integrates detection and correction into a single model. |
| Outcome: | The proposed model outperforms the state-of-the-art models on English and Chinese datasets. |
Exploring Multilingual Concepts of Human Values in Large Language Models: Is Value Alignment Consistent, Transferable and Controllable across Languages? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Prior research has revealed that certain abstract concepts are linearly represented as directions in the representation space of LLMs, predominantly centered around English. |
| Approach: | They extend previous research that shows certain abstract concepts are linearly represented as directions in LLMs, predominantly centered around English. |
| Outcome: | The proposed model can be used to align LLMs with human values, and it can generate toxic, untruthful, biased, and even illegal content. |
Predicting Foreign Language Usage from English-Only Social Media Posts (N18-2)
Copied to clipboard
| Challenge: | Social media is known for its multi-cultural and multilingual interactions, a natural product of which is code-mixing. |
| Approach: | They analyze 6 million tweets produced by 27 thousand multilingual users speaking 12 other languages besides English to build predictive models to infer non-English languages users speak exclusively from their tweets. |
| Outcome: | The proposed models are based on a corpus of 6 million tweets produced by 27 thousand multilingual users speaking 12 other languages besides English . they show that content, style and syntax are the most predictive of non-English languages that users speak on Twitter. |
Evaluating the Evaluation Metrics for Style Transfer: A Case Study in Multilingual Formality Transfer (2021.emnlp-main)
Copied to clipboard
| Challenge: | a lack of standardized and reliable methods for automatic evaluation hinders ST . prior work has employed as many as nine different automatic systems to rate formality alone . |
| Approach: | They evaluate automatic metrics on the oft-researched task of formality style transfer . they outline best practices for automatic evaluation in (formality) style transfer and identify models that correlate well with human judgments. |
| Outcome: | The proposed models correlate well with human judgments and are robust across languages. |
Developing and Utilizing a Large-Scale Cantonese Dataset for Multi-Tasking in Large Language Models (2025.findings-emnlp)
Copied to clipboard
Jiyue Jiang, Alfred Kar Yin Truong, Yanyu Chen, Qinghang Bao, Sheng Wang, Pengan Chen, Jiuming Wang, Lingpeng Kong, Yu Li, Chuan Wu
| Challenge: | Cantonese is considered a low-resource language due to the dominance of Mandarin . rich colloquial vocabulary of Cantone, English loanwords, and code-switching characteristics add to the complexity of corpus collection and processing. |
| Approach: | We collect Cantonese texts from open source corpora, Hong Kong-specific forums, Wikipedia . we refine the model through supervised fine-tuning on curated Cantonesian tasks . |
| Outcome: | The model achieves state-of-the-art (SOTA) performance on four Cantonese benchmarks. |
Cross-Lingual Abstract Meaning Representation Parsing (N18-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) research has focused on English . Qualitative analysis shows that the new parsers overcome structural differences between the languages. |
| Approach: | They propose to use an AMR parser for English and parallel corpora to learn AMR for Italian, Spanish, German and Chinese. |
| Outcome: | The proposed method overcomes structural differences between the target languages and requires no gold standard data. |
Exploiting Language Relatedness for Low Web-Resource Language Model Adaptation: An Indic Languages Study (2021.acl-long)
Copied to clipboard
Yash Khemchandani, Sarvesh Mehtani, Vaidehi Patil, Abhijeet Awasthi, Partha Talukdar, Sunita Sarawagi
| Challenge: | Recent research in multilingual language models (LMs) has demonstrated their ability to effectively handle multiple languages in a single model. |
| Approach: | They propose to exploit relatedness among languages in a language family to overcome corpora limitations of LRLs. |
| Outcome: | The proposed model exploits relatedness among languages in a language family to overcome corpora limitations for low web-resource languages. |
Assessing Monotonicity Reasoning in Dutch through Natural Language Inference (2023.findings-eacl)
Copied to clipboard
| Challenge: | a novel dataset for natural language inference (NLI) is used to study monotonicity reasoning in Dutch. |
| Approach: | They investigate monotonicity reasoning in Dutch using a novel dataset . they find that models struggle with downward entailing contexts . |
| Outcome: | The proposed dataset shows that models struggle with downward entailing contexts, and argue that this is due to a poor understanding of negation. |
SMHD-GER: A Large-Scale Benchmark Dataset for Automatic Mental Health Detection from Social Media in German (2023.findings-eacl)
Copied to clipboard
| Challenge: | Mental health problems are a challenge to our modern society, and their prevalence is predicted to increase worldwide. |
| Approach: | They propose a large-scale, carefully constructed dataset for MHC detection built on high-precision patterns and the approach proposed for English. |
| Outcome: | The proposed model leverages engineered (psycho-)linguistic features as well as BERT-German to facilitate further research and conduct extensive experiments. |
WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia (2021.eacl-main)
Copied to clipboard
| Challenge: | a new approach to extract parallel sentences from Wikipedia articles is proposed . the approach is based on multilingual sentence embeddings, but does not limit it to English . |
| Approach: | They propose to automatically extract parallel sentences from Wikipedia articles in 96 languages . they train neural MT baseline systems on the mined data and evaluate them on the TED corpus . |
| Outcome: | The proposed approach extracts parallel sentences from Wikipedia articles in 96 languages . the extracted sentences achieve strong BLEU scores for many language pairs . |
Testing Low-Resource Language Support in LLMs Using Language Proficiency Exams: the Case of Luxembourgish (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are used in research and society at large, but are mostly developed with English-speaking users in mind. |
| Approach: | They investigate the viability of language proficiency exams as evaluation tools for Luxembourgish . large models such as Claude and DeepSeek-R1 typically achieve high scores . |
| Outcome: | The proposed models can predict performance in Luxembourgish language tests. |
MlingConf: A Comprehensive Study of Multilingual Confidence Estimation on Large Language Models (2025.findings-acl)
Copied to clipboard
Boyang Xue, Hongru Wang, Rui Wang, Sheng Wang, Zezhong Wang, Yiming Du, Bin Liang, Wenxuan Zhang, Kam-Fai Wong
| Challenge: | Existing studies on LLM confidence estimations in languages other than English have been limited to English. |
| Approach: | They propose to use question-related language to prompt LLMs to assess their confidence in large language models. |
| Outcome: | The proposed model improves on question-related language prompts for LS tasks, while English exhibits notable linguistic dominance in confidence estimations. |
ManuSearch: Democratizing Deep Search in Large Language Models with a Transparent and Open Multi-Agent Framework (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing systems with opaque architectures are limiting deep search capabilities for web-augmented large language models. |
| Approach: | They propose a transparent and modular multi-agent framework to democratize deep search for LLMs. |
| Outcome: | The proposed framework outperforms open-source systems in deep reasoning tasks. |
MathMist: A Parallel Multilingual Benchmark Dataset for Mathematical Problem Solving and Reasoning (2026.findings-eacl)
Copied to clipboard
Mahbub E Sobhani, Md. Faiyaz Abdullah Sayeedi, Tasnim Mohiuddin, Md Mofijul Islam, Swakkhar Shatabda
| Challenge: | Existing benchmarks primarily focus on English or a narrow subset of high-resource languages, leaving significant gaps in assessing multilingual and cross-lingual mathematical reasoning. |
| Approach: | They propose a parallel multilingual benchmark for mathematical problem solving and reasoning that encompasses 2,890 parallel Bangla-English gold standard artifacts. |
| Outcome: | The proposed model encompasses 2,890 parallel Bangla-English gold standard artifacts, totaling 30K aligned question–answer pairs across thirteen languages, representing high-, medium-, and low-resource linguistic settings. |
Shallow Discourse Parsing for Under-Resourced Languages: Combining Machine Translation and Annotation Projection (2020.lrec-1)
Copied to clipboard
| Challenge: | Shallow Discourse Parsing (SDP) relies on large amounts of training data, which so far exists only for English. |
| Approach: | They propose to translate an existing English Penn Discourse TreeBank into German and use it to create a German corpus annotated for shallow discourse relations in the news domain. |
| Outcome: | The proposed corpus is annotated for shallow discourse relations in the (financial) news domain. |
DETECT: Determining Ease and Textual Clarity of German Text Simplifications (2026.eacl-long)
Copied to clipboard
| Challenge: | Current evaluation of German automatic text simplification relies on general-purpose metrics such as SARI, BLEU, and BERTScore. |
| Approach: | They propose a German-specific metric that holistically evaluates ATS quality across all three dimensions of simplicity, meaning preservation, and fluency. |
| Outcome: | The proposed metric achieves higher correlations with human judgments than widely used ATS metrics. |
Speaking Multiple Languages Affects the Moral Bias of Language Models (2023.findings-acl)
Copied to clipboard
Katharina Haemmerl, Bjoern Deiseroth, Patrick Schramowski, Jindřich Libovický, Constantin Rothkopf, Alexander Fraser, Kristian Kersting
| Challenge: | Pre-trained multilingual language models are often better on English than other languages . however, they are trained on varying amounts of data for each language . |
| Approach: | They apply the MORALDIRECTION framework to multilingual models and analyse their results . they find that PMLMs encode differing moral biases, but these do not correspond to cultural differences or commonalities in human opinions. |
| Outcome: | The proposed model captures moral norms from English and imposes them on other languages. |
Do LLM hallucination detectors suffer from low-resource effect? (2026.eacl-long)
Copied to clipboard
| Challenge: | a long line of work suggests that LLMs face issues along both dimensions . |
| Approach: | They investigate hallucination detectors' failure modes and their effects on the task accuracy of four LLMs and three halluciner detectors. |
| Outcome: | The models show impressive performance in high-resource languages like English but the performance degrades significantly in low-resourced languages like Bengali. |
Contextual String Embeddings for Sequence Labeling (C18-1)
Copied to clipboard
| Challenge: | Recent advances in language modeling have made it viable to model language as distributions over characters. |
| Approach: | They propose to leverage internal states of a trained character language model to produce a new type of word embeddings. |
| Outcome: | The proposed embeddings outperform the state-of-the-art on four classic sequence labeling tasks. |
CMU-MOSEAS: A Multimodal Language Dataset for Spanish, Portuguese, German and French (2020.emnlp-main)
Copied to clipboard
AmirAli Bagher Zadeh, Yansheng Cao, Simon Hessner, Paul Pu Liang, Soujanya Poria, Louis-Philippe Morency
| Challenge: | Existing datasets in multimodal language are limited and disproportionately affect native speakers of other languages . authors propose a large-scale dataset for Spanish, Portuguese, German and French . |
| Approach: | They propose a large-scale multimodal language dataset for Spanish, Portuguese, German and French. |
| Outcome: | The proposed dataset is the largest of its kind with 40,000 total labelled sentences . it covers a diverse set topics and speakers and carries supervision of 20 labels including sentiment, emotions, and attributes. |
Advancing Persian LLM Evaluation (2025.findings-naacl)
Copied to clipboard
Sara Bourbour Hosseinbeigi, Behnam Rohani, Mostafa Masoudi, Mehrnoush Shamsfard, Zahra Saaberi, Mostafa Karimi Manesh, Mohammad Amin Abbasi
| Challenge: | Existing evaluation approaches for large language models in low-resource languages like Persian lack comprehensive frameworks, limiting their ability to assess models’ performance over a wide range of tasks requiring considerable cultural and contextual knowledge. |
| Approach: | They propose to provide two new benchmarks to assess models' performance over a wide range of tasks requiring considerable cultural and contextual knowledge. |
| Outcome: | The proposed benchmarks challenge the current state-of-the-art models’ abilities in a variety of Persian language comprehension tasks while reducing data contamination while providing an accurate assessment of Persian LLMs. |
XLTime: A Cross-Lingual Knowledge Transfer Framework for Temporal Expression Extraction (2022.findings-naacl)
Copied to clipboard
| Challenge: | Temporal Expression Extraction (TEE) is essential for understanding time in natural language. |
| Approach: | They propose a framework for multilingual Temporal Expression Extraction that leverages pre-trained language models to prompt cross-language knowledge transfer from English to non-English languages. |
| Outcome: | The proposed framework outperforms the existing SOTA methods on French, Spanish, Portuguese, and Basque by large margins. |
A Variational Hierarchical Model for Neural Cross-Lingual Summarization (2022.acl-long)
Copied to clipboard
| Challenge: | Existing studies on cross-lingual summarization focus on pipeline methods or jointly training an end-to-end model through an auxiliary MT or MS objective. |
| Approach: | They propose a hierarchical model for the cross-lingual summarization task . the model is based on the conditional variational auto-encoder . |
| Outcome: | The proposed model generates better cross-lingual summaries than comparison models in the few-shot setting. |
Towards a Welsh Semantic Annotation System (L18-1)
Copied to clipboard
| Challenge: | Automatic semantic annotation of natural language data is an important task in Natural Language Processing. |
| Approach: | They develop a Welsh semantic annotation tool that can be used to analyze Welsh text . it uses Lancaster's USAS semantic classification scheme to tag words with semantic tags . |
| Outcome: | The proposed tool can cover up to 91.78% of words in Welsh text. |
X-LLaVA: Optimizing Bilingual Large Vision-Language Alignment (2024.findings-naacl)
Copied to clipboard
DongJae Shin, HyeonSeok Lim, Inho Won, ChangSu Choi, Minjun Kim, SeungWoo Song, HanGyeol Yoo, SangMin Kim, KyungTae Lim
| Challenge: | constructing multilingual data for large multimodal models presents its own set of challenges due to language diversity and complexity. |
| Approach: | They propose to use GPT4-V to construct multimodal training datasets using a text-only version of GPT4. |
| Outcome: | The proposed method performs well in Korean and English, surpassing existing methods. |
Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology (P19-1)
Copied to clipboard
| Challenge: | Gender stereotypes are manifest in most of the world's languages and are consequently propagated or amplified by NLP systems. |
| Approach: | They propose a method for converting between masculine-inflected and feminine-infflectes sentences in morphologically rich languages to reduce gender stereotyping by a factor of 2.5 without any sacrifice to grammaticality. |
| Outcome: | The proposed approach reduces gender stereotyping by 2.5 without any sacrifice to grammaticality. |
Cross-Lingual Machine Reading Comprehension (D19-1)
Copied to clipboard
| Challenge: | Existing work on machine reading comprehension task is focused on English, but there are few efforts on other languages due to the lack of large-scale training data. |
| Approach: | They propose a cross-lingual machine reading comprehension task for other languages . they propose cloze-style reading comprehension and various neural network approaches . |
| Outcome: | The proposed model improves reading comprehension performance of Chinese datasets over state-of-the-art systems by a large margin over existing systems. |
A Chinese Dataset for Evaluating the Safeguards in Large Language Models (2024.findings-acl)
Copied to clipboard
Yuxia Wang, Zenan Zhai, Haonan Li, Xudong Han, Shom Lin, Zhenxuan Zhang, Angela Zhao, Preslav Nakov, Timothy Baldwin
| Challenge: | a recent study has shown that large language models can produce harmful responses, exposing users to unexpected risks. |
| Approach: | They propose a dataset for the safety evaluation of Chinese LLMs in Mandarin Chinese . they extend the dataset to better identify false negative and false positive examples . |
| Outcome: | The proposed dataset is for the safety evaluation of Chinese LLMs, and is based on a Chinese dataset. |
BanglaRQA: A Benchmark Dataset for Under-resourced Bangla Language Reading Comprehension-based Question Answering with Diverse Question-Answer Types (2022.findings-emnlp)
Copied to clipboard
Syed Mohammed Sartaj Ekram, Adham Arik Rahman, Md. Sajid Altaf, Mohammed Saidul Islam, Mehrab Mustafy Rahman, Md Mezbaur Rahman, Md Azam Hossain, Abu Raihan Mostofa Kamal
| Challenge: | a lack of diverse and comprehensive question-answering datasets exists in under-resourced languages like Bangla. |
| Approach: | They propose a reading comprehension-based Bangla question-answering dataset . the dataset includes answerable and unanswerable questions covering four categories of questions . |
| Outcome: | The proposed dataset shows that it performs well as a training resource in high-resource languages. |
BioRo: The Biomedical Corpus for the Romanian Language (L18-1)
Copied to clipboard
| Challenge: | Biomedical text mining uses linguistic resources available in English, but for other languages such as Romanian, the access to language resources is not straight-forward. |
| Approach: | They present a biomedical corpus of the Romanian language, which is a valuable linguistic asset for biomedically text mining. |
| Outcome: | The proposed corpus will be made publicly available to the biomedical text mining community . the corpus is a reference corpus for the Romanian language . |
High-quality argumentative information in low resources approaches improve counter-narrative generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that fine-tuning improves the performance of language models . large language models generate acceptable texts in a number of scenarios, a study shows . |
| Approach: | They show that fine-tuning improves the task of hate speech counter-narrative generation . they provide a subset of arguments and a good base model is required for the fine-uning to have a positive impact. |
| Outcome: | The proposed model produces counter-narratives that are as satisfactory as the whole set. |
Neural Combinatory Constituency Parsing (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to constituency parsing are based on symbolic engineering, but they are simplified by their adaptive distributed representation. |
| Approach: | They propose two fast combinatory models for constituency parsing: binary and multibranching. |
| Outcome: | The proposed models achieve an F1 score of 92.54 on Penn Treebank, speeding at 1327.2 sents/sec. |
Retrieval Augmented Instruction Tuning for Open NER with Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies have focused on integrating large language models (LLMs) with information extraction (IE) however, the best approach to incorporate information with LLMs for IE remains an open question. |
| Approach: | They propose to use a Chinese IT dataset to perform RA-IT for IE . they use semantically similar examples from the training dataset as the context . |
| Outcome: | The proposed approach is evaluated in English and Chinese scenarios. |
Detecting dementia in Mandarin Chinese using transfer learning from a parallel corpus (N19-1)
Copied to clipboard
| Challenge: | Existing methods for automatic detection of Alzheimer's disease (AD) are limited by a lack of data. |
| Approach: | They propose a method to learn a correspondence between independently engineered lexicosyntactic features in two languages, using a large parallel corpus of out-of-domain movie dialogue data. |
| Outcome: | The proposed method outperforms both unilingual and machine translation-based baselines in Mandarin Chinese and is the first to transfer feature domains in detecting cognitive decline. |
The Arabic Parallel Gender Corpus 2.0: Extensions and Analyses (2022.lrec-1)
Copied to clipboard
| Challenge: | Gender bias in natural language processing (NLP) applications has been receiving increasing attention, largely due to the lack of datasets and resources. |
| Approach: | They propose a corpus for gender identification and rewriting in contexts involving one or two target users with independent grammatical gender preferences. |
| Outcome: | The proposed corpus expands on Habash et al.'s Arabic Parallel Gender Corpus (APGC) by adding second person targets and increasing the total number of sentences over 6.5 times, reaching over 590K words. |
CEASE, a Corpus of Emotion Annotated Suicide notes in English (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing studies on suicide notes have not explored the topic of emotion detection. |
| Approach: | They develop a fine-grained emotion annotated corpus of suicide notes in English and use it to perform emotion detection on a curated dataset. |
| Outcome: | The proposed model performs emotion detection on a curated dataset of 205 suicide notes in English. |
Science Across Languages: Assessing LLM Multilingual Translation of Scientific Papers (2026.findings-eacl)
Copied to clipboard
| Challenge: | a large number of scientific journals are published exclusively in English . this creates barriers for non-native English speakers to access scientific knowledge . |
| Approach: | They propose a way to translate scientific articles while preserving native JATS XML formatting. |
| Outcome: | The proposed approach shows that the key scientific details are accurately conveyed. |
Assessing Multilingual Fairness in Pre-trained Multimodal Representations (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent pre-trained multimodal models have shown exceptional capabilities towards connecting images and natural language. |
| Approach: | They propose two new fairness notions for pre-trained multimodal models that consider language as the fairness recipient. |
| Outcome: | The proposed models can be generalized to multilingualism by cross-lingual alignment . the results show that the models are individually fair across languages . |
Modal Dependency Parsing via Language Model Priming (2022.naacl-main)
Copied to clipboard
| Challenge: | modal dependency parsing is a task of parse a text into its modal dependence structure . the root node of an MDS is always the author of a document, the ultimate source of information sources . |
| Approach: | They propose a modal dependency parser based on priming pre-trained language models and evaluate it on two data sets. |
| Outcome: | The proposed parser improves on two data sets. |
PPL-MCTS: Constrained Textual Generation Through Discriminator-Guided MCTS Decoding (2022.naacl-main)
Copied to clipboard
| Challenge: | Large language models (LM) based on transformers generate plausible long texts . a discriminator-guided approach allows to apply constraints more finely and dynamically. |
| Approach: | They propose to use a discriminator-guided approach to generate constrained texts without fine-tuning the LM. |
| Outcome: | The proposed method is easier and cheaper to train than fine-tuning the LM. |
Fine-grained Semantic Textual Similarity for Serbian (L18-1)
Copied to clipboard
| Challenge: | Semantic textual similarity (STS) is a task of assigning a numerical score to short texts based on the level of semantic equivalence between them. |
| Approach: | They propose to annotate Serbian STS dataset with fine-grained similarity scores . they propose a supervised bag-of-words model that combines part-of speech weighting with term frequency weighting . |
| Outcome: | The proposed model outperforms existing models on the Serbian STS News Corpus . the proposed model is based on a new morphologically rich language . |
Multilingual Summarization with Factual Consistency Evaluation (2023.findings-acl)
Copied to clipboard
| Challenge: | Abstractive summarization models generate factually inconsistent summaries, reducing their utility for real-world applications. |
| Approach: | They propose to use data filtering and controlled generation to detect hallucinations in machine generated summaries. |
| Outcome: | The proposed models detect factual inconsistencies in machine generated summaries, but they focus on English only. |
LaCoMSA: Language-Consistency Multilingual Self-Alignment with Latent Representation Rewarding (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing multilingual alignment methods mitigate these issues but rely on external supervision, such as translation systems or English-biased signal. |
| Approach: | They propose a preference optimization framework that leverages an LLM’s own latent representations as intrinsic supervision signals and rewards lower-resource language outputs based on their alignment with high-resourced (English) counterparts in the "semantic hub". |
| Outcome: | The proposed framework improves a Llama 3 8B model multilingual win rates by up to 6.8% absolute (55.0% relative) on X-AlpacaEval and achieves consistent gains across benchmarks and models. |
From Toxicity in Online Comments to Incivility in American News: Proceed with Caution (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing tools for quantifying incivility online, in news and in congressional debates are inadequate for the analysis of incivility in news. |
| Approach: | They develop a Jigsaw Perspective API to quantify incivility in news . they show that toxicity models are inadequate for the analysis of incivility in news. |
| Outcome: | The Jigsaw Perspective API detects incivility on a corpus of American news articles. |
ERNIE-Doc: A Retrospective Long-Document Modeling Transformer (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for document-level language pretraining are not suitable for long documents due to their quadratically increasing memory and time consumption. |
| Approach: | They propose a document-level language pretraining model based on Recurrence Transformers. |
| Outcome: | The proposed model outperforms existing models on language understanding tasks. |
Robustness of Multi-Source MT to Transcription Errors (2023.findings-acl)
Copied to clipboard
| Challenge: | In multilingual settings, the same content may be available in various languages via simultaneous interpreting, dubbing or subtitling. |
| Approach: | They hypothesize that leveraging multiple sources will improve translation quality if the sources complement one another in terms of correct information they contain. |
| Outcome: | The proposed method is robust to speech recognition errors on a 10-hour ESIC corpus. |
Compound Probabilistic Context-Free Grammars for Grammar Induction (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to grammar induction have resorted to manually-engineered features and auxiliary objectives to induce the desired structures. |
| Approach: | They propose a formalization of the grammar induction problem that models sentences as being generated by a compound probabilistic context free grammar. |
| Outcome: | Experiments on English and Chinese show that the proposed approach is more efficient than other methods. |
The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Text (2024.findings-naacl)
Copied to clipboard
| Challenge: | a new study examines the effects of training language models on synthetic data generated by their predecessors. |
| Approach: | They propose to use recursive finetuning techniques to assess linguistic diversity of models. |
| Outcome: | The proposed metrics show a decrease in diversity of model outputs through successive iterations, especially for tasks demanding high levels of creativity. |
Enriching Biomedical Knowledge for Low-resource Language Through Large-scale Translation (2023.eacl-main)
Copied to clipboard
| Challenge: | Biomedical data and benchmarks are highly valuable but limited in low-resource languages such as English. |
| Approach: | They propose a translation model in Vietnamese that trains a pretrained Encoder-Decoder Transformer model on 20 million translated abstracts. |
| Outcome: | The proposed model can translate and produce both pretrained and supervised biomedical data in two biomedically important domains. |
Respond in my Language: Mitigating Language Inconsistency in Response Generation based on Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) show strong instruction understanding ability across multiple languages, but are easily biased towards English in instruction tuning. |
| Approach: | They propose to use a model with Pseudo-Inconsistent Penalization to prevent the model from generating English responses when given non-English language prompts during training and prior Enhanced decoding to improve the language consistency of the model. |
| Outcome: | The proposed methods significantly improve the language consistency of the model without multilingual data. |
Edit-Wise Preference Optimization for Grammatical Error Correction (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been successful in grammatical error correction (GEC) but their strengths have yet to be fully demonstrated in GEC . |
| Approach: | They propose a method to optimize grammatical errors by assigning higher reward weights to edit tokens during preference optimization. |
| Outcome: | The proposed method outperforms baselines on English and Chinese datasets and achieves state-of-the-art performance. |
Building a Japanese Document-Level Relation Extraction Dataset Assisted by Cross-Lingual Transfer (2024.lrec-main)
Copied to clipboard
| Challenge: | Document-level Relation Extraction (DocRE) is the task of extracting all semantic relationships from a document. |
| Approach: | They propose to transfer an English document to Japanese to promote DocRE in other languages. |
| Outcome: | The proposed model reduces the human edit steps by 50% compared with the previous approach. |
A Vietnamese Dataset for Evaluating Machine Reading Comprehension (2020.coling-main)
Copied to clipboard
| Challenge: | despite the lack of benchmark datasets for Vietnamese, there are few studies on machine reading comprehension (MRC) . MRC is an essential core for a range of natural language processing applications such as search engines and intelligent agents. |
| Approach: | They propose to use Vietnamese Question Answering Dataset to evaluate machine reading comprehension in Vietnamese . they use over 23,000 human-generated question-answer pairs based on 5,109 Vietnamese articles . |
| Outcome: | The proposed dataset includes over 23,000 human-generated question-answer pairs based on 5,109 passages of 174 Vietnamese articles from Wikipedia. |
Embedding Style Beyond Topics: Analyzing Dispersion Effects Across Different Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Using a literary corpus that alternates between topics and styles, we compare language models across French and English. |
| Approach: | They analyze how writing style affects embedding spaces across multiple language models . they use a literary corpus that alternates between topics and styles to compare their results . |
| Outcome: | The proposed model is based on two established literary works in French and English. |
Multilingual AMR Parsing with Noisy Knowledge Distillation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) parsing is a broad-coverage semantic formalism that encodes the meaning of a sentence as a rooted, directed, and labeled graph. |
| Approach: | They propose to use existing English parser to learn and improve multilingual AMR parsers . their results show that noisy input and precise output are key to successful distillation . |
| Outcome: | The proposed model outperforms the current state-of-the-art English-only parser on four different languages. |
Improving English-Arabic Transliteration with Phonemic Memories (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing neural approaches to transliterate names from English to Arabic are limited and focus on leveraging the phonemic association between English and Arabic. |
| Approach: | They propose a model for English-Arabic transliteration using a memory module modeling the phonemic association between English and Arabic to guide the transliterations process. |
| Outcome: | The proposed model improves on EANames corpus, which better represents names in the general public than linked Wikipedia entries that are always names of famous people. |
Topic-Aware Neural Keyphrase Generation for Social Media Language (P19-1)
Copied to clipboard
| Challenge: | Existing methods to extract words from source posts to form keyphrases do not exploit latent topics. |
| Approach: | They propose a sequence-to-sequence-based neural keyphrase generation framework . it allows absent keyphrases to be created, and it allows joint modeling of latent topic representations . |
| Outcome: | The proposed model outperforms extraction and generation models without exploiting latent topics. |
Information Representation Fairness in Long-Document Embeddings: The Peculiar Interaction of Positional and Language Bias (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that embedding models exhibit systematic positional and language biases when documents are longer and consist of multiple segments. |
| Approach: | They propose a permutation-based evaluation framework to quantify embedding biases . they propose an inference-time attention calibration method that redistributes attention more evenly across document positions . |
| Outcome: | The proposed framework reduces the positional and language biases in embedding models . the proposed framework improves the discoverability of later segments . |
CHEF: A Pilot Chinese Dataset for Evidence-Based Fact-Checking (2022.naacl-main)
Copied to clipboard
| Challenge: | CHEF dataset provides evidence retrieval over non-English claims . e-fact-checking is a time-consuming task, which can take journalists several hours or days. |
| Approach: | They construct a dataset of 10K real-world claims that is based on annotated evidence retrieved from the Internet. |
| Outcome: | The proposed dataset provides evidence retrieval as a latent variable and can be used to train and reason over non-English claims. |
Completely Modular Fine-tuning for Dynamic Language Adaptation (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies on multilingual fine-tuning with a fixed set of languages lack dynamic adaptability to new languages. |
| Approach: | They propose a modular fine-tuning pipeline that enables dynamic language adaptation for LLMs by first training English-centric adapters for each language separately and then merging them for arbitrary-direction translation. |
| Outcome: | The proposed pipeline achieves 86% performance over traditional fine-tuning on four languages, while training only 0.1% parameters and relying on English as a bridge language without catastrophic forgetting. |
How Well Do LLMs Handle Cantonese? Benchmarking Cantonese Capabilities of Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Cantonese has scant representation in NLP research, especially compared to other languages from similarly developed regions. |
| Approach: | They propose to evaluate Cantonese LLM performance in factual generation, mathematical logic, complex reasoning, and general knowledge in Cantonesian. |
| Outcome: | The proposed models will evaluate Cantonese's performance in factual generation, mathematical logic, complex reasoning, and general knowledge in Cantone. |
Attention Can Reflect Syntactic Structure (If You Let It) (2021.eacl-main)
Copied to clipboard
| Challenge: | a recent study has attempted to decode linguistic structure from the Transformer . but, much of the work focused on English, a language with rigid word order and a lack of inflectional morphology. |
| Approach: | They propose to fine-tune a feature encoder for BERT to learn linguistic structure from its multi-head attention mechanism. |
| Outcome: | The proposed model can decode full trees above baseline accuracy from single attention heads across languages. |
Cross-Lingual UMLS Named Entity Linking using UMLS Dictionary Fine-Tuning (2022.findings-acl)
Copied to clipboard
| Challenge: | a new method for named entity linking is being developed in the field of public health . it uses an offline unsupervised construction of a translated dictionary and a pre-trained transformer language model to filter candidates according to context. |
| Approach: | They propose a method for mapping mentions in a source language to UMLS concepts . they extend an offline unsupervised translation of a translated UMLS dictionary . |
| Outcome: | The proposed approach achieves state-of-the-art on the Hebrew Camoni corpus and English datasets. |
Huge Automatically Extracted Training-Sets for Multilingual Word SenseDisambiguation (L18-1)
Copied to clipboard
| Challenge: | Word Sense Disambiguation is a crucial task in Natural Language Processing . supervised systems need to be trained on word-by-word basis, a problem that is beyond reach for resource-rich languages like English. |
| Approach: | They release six large-scale sense-annotated datasets in multiple languages to pave the way for supervised multilingual Word Sense Disambiguation. |
| Outcome: | The results show that large-scale sense annotations can be used as training sets for supervised systems. |
Continually Detection, Rapidly React: Unseen Rumors Detection Based on Continual Prompt-Tuning (2022.coling-1)
Copied to clipboard
| Challenge: | Existing rumor detection models assume the same training and testing distributions and can not cope with the continuously changing social network environment. |
| Approach: | They propose a Continual Prompt-Tuning RD framework which avoids catastrophic forgetting of upstream tasks during sequential task learning and enables bidirectional knowledge transfer between domain tasks. |
| Outcome: | The proposed framework avoids catastrophic forgetting (CF) of upstream tasks during sequential task learning and enables bidirectional knowledge transfer between domain tasks. |
Studying Taxonomy Enrichment on Diachronic WordNet Versions (2020.coling-main)
Copied to clipboard
| Challenge: | Ontologies, taxonomies and thesauri are used in many NLP tasks but are often not maintained. |
| Approach: | They propose methods for taxonomy enrichment in a resource-poor setting . they also create novel datasets for training and evaluating taxonomies . |
| Outcome: | The proposed methods are applicable to English and Russian datasets and can be used in other languages. |
Exploring the Representation of Word Meanings in Context: A Case Study on Homonymy and Synonymy (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models that represent different senses of words in context are not accurate for polysemous words. |
| Approach: | They propose a multilingual dataset that evaluates the ability of models to accurately represent different lexical-semantic relations such as homonymy and synonymy. |
| Outcome: | The proposed models can disambiguate homonyms in context, but fail to represent words with different senses when occurring in similar sentences. |
Counter Turing Test (CT2): Investigating AI-Generated Text Detection for Hindi - Ranking LLMs based on Hindi AI Detectability Index (ADI_hi) (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a growing number of large language models are being used to detect AI-generated text . a recent study has found that some techniques to bypass detection are fragile . |
| Approach: | They propose to use 26 LLMs to evaluate their proficiency in generating Hindi text . they propose to introduce a Hindi AI Detectability Index to assess and rank LLM models based on their detectability levels. |
| Outcome: | The proposed methods are effective in English, but struggle in Hindi . the proposed methods show that they are susceptible to fragility . |
SciPar: A Collection of Parallel Corpora from Scientific Abstracts (2022.lrec-1)
Copied to clipboard
Dimitrios Roussis, Vassilis Papavassiliou, Prokopis Prokopidis, Stelios Piperidis, Vassilis Katsouros
| Challenge: | SciPar is a collection of parallel corpora created from openly available metadata of bachelor theses, master theses and doctoral dissertations hosted in institutional repositories, digital libraries and national archives. |
| Approach: | They propose to harvest and process openly available metadata from repositories to extract bilingual titles and abstracts from scientific publications. |
| Outcome: | The proposed corpora could be useful for cross-lingual plagiarism detection or adapting Machine Translation systems for translation of scientific texts and academic writing in general. |
Arabic Dataset for LLM Safeguard Evaluation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing studies on large language models have focused on English, but the safety of LLMs in Arabic remains under-explored. |
| Approach: | They propose to use Arabic-region-specific questions to evaluate LLMs' safety . they use a dual-perspective evaluation framework to examine differences between LLM responses . |
| Outcome: | The proposed framework assesses the LLM responses from both governmental and opposition viewpoints. |
Document-Level Event Factuality Identification via Adversarial Neural Network (N19-1)
Copied to clipboard
| Challenge: | Document-level event factuality identification is crucial for discourse understanding in NLP . identifying document-level factual of events requires comprehensive understanding of documents . |
| Approach: | They propose to construct a corpus annotated with document- and sentence-level event factuality information on English and Chinese texts. |
| Outcome: | The proposed model outperforms baselines on the constructed corpus. |
On the Calibration of Massively Multilingual Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Massively Multilingual Language Models (MMLMs) have gained popularity due to their effectiveness in cross-lingual transfer. |
| Approach: | They investigate how well calibrated MMLMs are with respect to confidence . they find that calibration methods like temperature scaling and label smoothing improve calibration . |
| Outcome: | The proposed models are able to generalize in languages unseen during fine-tuning, but they are not reliable across languages. |
MLaKE: Multilingual Knowledge Editing Benchmark for Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies on knowledge editing focus on monolingual scenarios, neglecting the complexities presented by multilingual contexts and multi-hop reasoning. |
| Approach: | They propose a benchmark to evaluate the adaptability of multilingual knowledge editing methods. |
| Outcome: | The proposed benchmark evaluates the adaptability of multilingual knowledge editing methods across five languages. |
FlauBERT: Unsupervised Language Model Pre-training for French (2020.lrec-1)
Copied to clipboard
Hang Le, Loïc Vial, Jibril Frej, Vincent Segonne, Maximin Coavoux, Benjamin Lecouteux, Alexandre Allauzen, Benoit Crabbé, Laurent Besacier, Didier Schwab
| Challenge: | Language models are a key step to achieve state-of-the-art results in many different Natural Language Processing (NLP) tasks. |
| Approach: | They propose to use a language model that is pre-trained on a large and heterogeneous French corpus to train continuous word representations. |
| Outcome: | The proposed model outperforms existing models on a large and heterogeneous French corpus. |
Don’t Change Me! User-Controllable Selective Paraphrase Generation (2021.eacl-main)
Copied to clipboard
| Challenge: | a new technique allows paraphrase generation to be user-controlled . a user looking for cheap hotels in New York would not find the other answer helpful . |
| Approach: | They propose a method that provides a user with explicit tags that can be placed around any arbitrary segment of text to mean "don't change me!" they propose allowing user-controllable paraphrase generation by fine-tuning model that exhibits this behavior . |
| Outcome: | The proposed technique is language agnostic and tested in English and Chinese. |
Probing for idiomaticity in vector space models (2021.eacl-main)
Copied to clipboard
| Challenge: | Contextualised word representation models are used to represent idiomaticity in language. |
| Approach: | They propose probing measures to assess if some of the expected linguistic properties of noun compounds are readily available in some standard and widely used representations. |
| Outcome: | The proposed models show that idiomaticity is not yet accurately represented by contextualised models. |
It’s All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoning (2021.findings-acl)
Copied to clipboard
| Challenge: | gilbert et al.: commonsense reasoning is a key problem in natural language processing but its capabilities are still unstudied. gilland eetal.: a new approach to commonsensible reasoning is needed to solve the problem. |
| Approach: | They propose a method which trains a linear classifier with weights of multi-head attention as features and a multilingual Winograd Schema corpus to measure cross-lingual generalization ability. |
| Outcome: | The proposed approach performs competitively with recent approaches even when applied to other languages in a zero-shot manner. |
End-to-End Lexically Constrained Machine Translation for Morphologically Rich Languages (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to enforce word forms in translations struggle to make them agree with the rest of the output. |
| Approach: | They propose to train neural machine translation models with lemmatized constraints to infer correct word inflection. |
| Outcome: | The proposed model reduces errors in translation of constrained terms in automatic and manual evaluations on English-Czech language pairs. |
ViSoBERT: A Pre-Trained Language Model for Vietnamese Social Media Text Processing (2023.emnlp-main)
Copied to clipboard
| Challenge: | English and Chinese have seen the strong development of transformer-based language models for natural language processing tasks. |
| Approach: | They present a monolingual pre-trained language model for Vietnamese social media texts . they explore emotion recognition, hate speech detection, sentiment analysis, spam reviews detection . |
| Outcome: | The proposed model outperforms the existing models on Vietnamese social media tasks with fewer parameters. |
Multimodal neural pronunciation modeling for spoken languages with logographic origin (D18-1)
Copied to clipboard
| Challenge: | Graphemes of most languages encode pronunciation, though some are more explicit than others . pronunciation modeling in logographic languages requires decomposing logographs into subunits . |
| Approach: | They propose a multimodal approach to predict pronunciation of Cantonese logographic characters using neural networks. |
| Outcome: | The proposed framework improves performance by 18.1% and 25.0% respectively to unimodal and multimodal baselines. |
Deep Inside-outside Recursive Autoencoder with All-span Objective (2020.coling-main)
Copied to clipboard
| Challenge: | Existing neural approaches for constituency parsing are limited for low-resource languages and domains. |
| Approach: | They extend the training objective of DIORA by making use of all spans instead of only leaf-level spans. |
| Outcome: | The proposed model improves on two languages and provides better parsing accuracy than the original model. |
Korean TimeBank Including Relative Temporal Information (L18-1)
Copied to clipboard
| Challenge: | Temporal information extraction is one of the important research fields in natural language processing. |
| Approach: | They propose a concept of relative temporal information and supplement a Korean annotation language to represent new relative expressions and extend an annotated dataset through the revised language. |
| Outcome: | The proposed language can be used to represent relative temporal information and extend an annotated dataset, Korean TimeBank, through the revised language. |
Compositional Demographic Word Embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Word embeddings are usually derived from corpora containing text from many individuals . however, they cannot account for user-specific word preferences, such as using the same word in different ways across contexts. |
| Approach: | They propose a new form of personalized word embeddings that use demographic-specific word representations derived compositionally from full or partial demographic information for a user. |
| Outcome: | The proposed representations outperform generic representations on two English language tasks. |
Learning from Child-directed Speech in Two-language Scenarios: A French-English Case-Study (2026.findings-eacl)
Copied to clipboard
| Challenge: | a systematic study of compact language models with limited computational resources is challenging for many research contexts and real-world applications. |
| Approach: | They extend BabyBERTa to English-French scenarios under strictly sizematched data conditions. |
| Outcome: | The proposed model extends to English-French scenarios under sizematched data conditions . the results show context-dependent effects of multilingual training . |
Multilingual Constituency Parsing with Self-Attention and Pre-Training (P19-1)
Copied to clipboard
| Challenge: | a range of pre-training conditions can be used for constituency parsing, but large model sizes make it expensive to train separate models for each language. |
| Approach: | They compare the benefits of no pre-training, fastText, ELMo, and BERT for English . they also find that pre- training is beneficial across all 11 languages tested . |
| Outcome: | The proposed model outperforms fastText, ELMo, and BERT for English . but large model sizes make it expensive to train separate models for each language . |
MEENA (PersianMMMU): Multimodal-Multilingual Educational Exams for N-level Assessment (2026.findings-eacl)
Copied to clipboard
Omid Ghahroodi, Arshia Hemmat, Marzia Nouri, Seyed Mohammad Hadi Hosseini, Doratossadat Dastgheib, Mohammad Vali Sanian, Alireza Sahebi, Reihaneh Zohrabi, Mohammad Hossein Rohban, Ehsaneddin Asgari, Mahdieh Soleymani Baghshah
| Challenge: | Recent advances in large vision-language models have primarily focused on English, with limited attention given to other languages. |
| Approach: | They propose a dataset to evaluate Persian VLMs across scientific, reasoning, and human-level understanding tasks. |
| Outcome: | The proposed model performs well across scientific reasoning, reasoning, and human-level understanding tasks in Persian and English. |
Better, Faster, Stronger Sequence Tagging Constituent Parsers (N19-1)
Copied to clipboard
| Challenge: | Existing efforts to speed up constituent parsing have focused on chart-based or shift-reduce parsers. |
| Approach: | They propose to use auxiliary losses and sentence-level fine-tuning to mitigate greedy decoding issues. |
| Outcome: | The proposed model surpasses the performance of sequence tagging constituent parsers on the English and Chinese Penn Treebank datasets and reduces their parsing time even further. |
Beyond Understanding: Evaluating the Pragmatic Gap in LLMs’ Cultural Processing of Figurative Language (2026.eacl-long)
Copied to clipboard
| Challenge: | Using figurative language as a proxy for cultural nuance and local knowledge, large language models struggle with connotative meaning. |
| Approach: | They evaluate large language models' ability to process culturally grounded language . they use figurative language as a proxy for cultural nuance and local knowledge . |
| Outcome: | The proposed models can understand and use figurative expressions that encode local knowledge and social nuance. |
Singlish Message Paraphrasing: A Joint Task of Creole Translation and Text Normalization (2022.coling-1)
Copied to clipboard
| Challenge: | Existing computational approaches to translate languages or creoles back to standard English are challenging . lexical level normalization, syntactic level editing, and semantic level rewriting are key to a successful translation task. |
| Approach: | They propose a computational task to parse Singlish into English using its dialects . they propose to use a dataset to normalize and edit the text to improve translation . |
| Outcome: | The proposed model can improve translation performance and improve stance detection. |
ClozEx: A Task toward Generation of English Cloze Explanation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing tasks and datasets specifically designed for generating language learner explanations for cloze questions are lacking . clozing questions are used to assess language proficiency and enhance language learning . |
| Approach: | They propose a task ClozEx to generate explanations for cloze questions in LA . they use a curated dataset of clozing questions paired with explanations . |
| Outcome: | The proposed task generates fluent explanations for cloze questions in English as a second language learners. |
SongComposer: A Large Language Model for Lyric and Melody Generation in Song Composition (2025.acl-long)
Copied to clipboard
Shuangrui Ding, Zihan Liu, Xiaoyi Dong, Pan Zhang, Rui Qian, Junhao Huang, Conghui He, Dahua Lin, Jiaqi Wang
| Challenge: | Creating lyrics and melodies in symbolic format requires expert knowledge of melody and an advanced understanding of lyrics. |
| Approach: | They introduce SongComposer, a music-specialized large language model that can create symbolic lyrics and melodies following instructions. |
| Outcome: | The proposed model outperforms existing models in symbolic song composition tasks. |
Cards Against Contamination: TCG-Bench for Difficulty-Scalable Multilingual LLM Reasoning (2026.findings-eacl)
Copied to clipboard
| Challenge: | Recent studies find 25-50% of evaluation datasets appear in training corpora . contamination hinders the possibility to differentiate memorization and reasoning skills. |
| Approach: | They propose a two-player trading card game that is contaminated by a public engine and hidden card implementations to prevent benchmark saturation. |
| Outcome: | The proposed benchmark is based on a new two-player trading card game similar to Magic: The Gathering. |
Learning Implicit Text Generation via Feature Matching (2020.acl-main)
Copied to clipboard
Inkit Padhi, Pierre Dognin, Ke Bai, Cícero Nogueira dos Santos, Vijil Chenthamarakshan, Youssef Mroueh, Payel Das
| Challenge: | Generative feature matching networks are an approach for training implicit generative models for images . a novel formulation of GFMN for unconditional sequence generation is proposed . |
| Approach: | They propose a new GFMN formulation that performs token level feature matching on pre-trained neural networks. |
| Outcome: | The proposed method outperforms adversarial approaches for text generation and style transfer. |
Studying the Inductive Biases of RNNs with Synthetic Variations of Natural Languages (N19-1)
Copied to clipboard
| Challenge: | Recent studies have identified both strengths and limitations of recurrent neural networks (RNNs) in applied natural language processing tasks. |
| Approach: | They propose a paradigm that addresses typological differences between languages . they create synthetic versions of English and train them to predict agreement features . |
| Outcome: | The proposed model improves on predicting agreement with subject and object, suggesting that RNNs have a recency bias. |
Graph-Assisted Culturally Adaptable Idiomatic Translation for Indic languages (2025.findings-acl)
Copied to clipboard
| Challenge: | a single source idiom can have multiple target-language equivalents depending on cultural references and contextual variations. |
| Approach: | They propose an adaptive graph neural network-based method that learns intricate mappings between idiomatic expressions and generalizes to both seen and unseen nodes during training. |
| Outcome: | The proposed method improves translation quality even in resource-constrained settings, facilitating improved idiomatic translation in smaller models. |
Cross-lingual Transfer or Machine Translation? On Data Augmentation for Monolingual Semantic Textual Similarity (2024.lrec-main)
Copied to clipboard
| Challenge: | Using labeled NLI datasets for learning sentence embeddings leads to improved performance for natural language understanding tasks. |
| Approach: | They compare two data augmentation techniques for learning better sentence embeddings . they use a cross-lingual transfer technique that exploits English resources as training data to yield non-English sentence embeds as zero-shot inference . |
| Outcome: | The proposed techniques yield better performance on Japanese and Korean sentences. |
Structural Supervision Improves Few-Shot Learning and Syntactic Generalization in Neural Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have not investigated the relationship between a token's frequency in the training corpus and syntactic properties models learn about it. |
| Approach: | They develop controlled experiments that probe models’ syntactic nominal number and verbal argument structure generalizations for tokens seen as few as two times during training. |
| Outcome: | The proposed models can make syntactic generalizations for tokens seen as few as two times during training and transfer them to transformed contexts. |
CSSWiki: A Chinese Sentence Simplification Dataset with Linguistic and Content Operations (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing datasets for sentence simplification focus on English, but limited in Chinese . SS tasks are aimed at improving readability and making sentences more accessible for readers . |
| Approach: | They propose an open-source dataset for Chinese sentence simplification based on Wikipedia . they analyze differences in annotation scheme and data statistics between datasets . |
| Outcome: | The proposed dataset contains 1.6k source sentences paired with their simplified versions. |
A Danish FrameNet Lexicon and an Annotated Corpus Used for Training and Evaluating a Semantic Frame Classifier (L18-1)
Copied to clipboard
| Challenge: | a Danish FrameNet is a lexicon based on the Danish Thesaurus . it is significantly faster than building a new one from scratch . |
| Approach: | They propose a way to efficiently compile a Danish FrameNet based on the Danish Thesaurus . they present the corresponding corpus annotations of frames and roles and show how this can be used for a semantic frame classifier . |
| Outcome: | The proposed approach is faster than building a lexicon from scratch. |
How to Make the Most of LLMs’ Grammatical Knowledge for Acceptability Judgments (2025.naacl-long)
Copied to clipboard
Yusuke Ide, Yuto Nishida, Justin Vasselli, Miyu Oba, Yusuke Sakai, Hidetaka Kamigaito, Taro Watanabe
| Challenge: | Conventional approaches compare sentence probabilities directly, but large language models (LLMs) provide nuanced evaluation methods using prompts and templates. |
| Approach: | They propose to derive acceptability judgments from large language models using prompts and templates to comprehensively evaluate their grammatical knowledge. |
| Outcome: | The proposed methods excel in different linguistic phenomena, suggesting they access different aspects of the LLMs’ grammatical knowledge. |
DLAMA: A Framework for Curating Culturally Diverse Facts for Probing the Knowledge of Pretrained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | a few benchmarking datasets have been released to evaluate the factual knowledge of pretrained language models. |
| Approach: | They propose a framework for curating factual triples from Wikidata that are culturally diverse. |
| Outcome: | The proposed framework is built of factual triples from three pairs of contrasting cultures with 78,259 triples. |
What’s so special about BERT’s layers? A closer look at the NLP pipeline in monolingual and multilingual models (2020.findings-emnlp)
Copied to clipboard
| Challenge: | In addition, information on part-of-speech tagging is spread over different parts of the network and the pipeline might not be as neat as it seems. |
| Approach: | They propose to probe Dutch BERT-based model and multilingual BERT model for Dutch NLP tasks to see if this holds true for other languages. |
| Outcome: | The proposed model is based on a Dutch model and a multilingual model for Dutch NLP tasks. |
Combining Discourse Markers and Cross-lingual Embeddings for Synonym–Antonym Classification (N19-1)
Copied to clipboard
| Challenge: | Recent work shows that distributional semantic approaches have difficulty distinguishing between synonyms and antonyms. |
| Approach: | They propose to use monolingual distributional information available in a target language to transfer supervision to other languages using cross-lingual word embeddings. |
| Outcome: | The proposed method improves the transfer of monolingual distributional information to other languages using co-occurrences with discourse markers indicative of antonymy. |
MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answering (2025.findings-acl)
Copied to clipboard
Jingqun Tang, Qi Liu, Yongjie Ye, Jinghui Lu, Shu Wei, An-Lan Wang, Chunhui Lin, Hao Feng, Zhen Zhao, Yanjie Wang, Yuliang Liu, Hao Liu, Xiang Bai, Can Huang
| Challenge: | Text-Centric Visual Question Answering (TEC-VQA) is a text-centric visual task understanding tool. |
| Approach: | They introduce a benchmark that features human expert annotations across 9 languages . they prioritize the text in question-answer pairs while disregarding visual text in images . |
| Outcome: | The proposed benchmarks prioritize the text in question-answer pairs while disregarding visual text in images. |
Hi-GEC: Hindi Grammar Error Correction in Low Resource Scenario (2025.coling-main)
Copied to clipboard
| Challenge: | Automated Grammatical Error Correction (GEC) is a scarcely explored low-resource language . a recent study focused on English, but it focused on Hindi, which presents unique challenges due to its complex syntax and intricate morphology. |
| Approach: | They propose to use a human-edited dataset to generate Hindi GEC data . they also investigate round trip translation using diverse languages for the technique . |
| Outcome: | The proposed method outperforms other methods in Hindi, showing that it is highly efficient. |
AcrosticSleuth: Probabilistic Identification and Ranking of Acrostics in Multilingual Corpora (2025.findings-naacl)
Copied to clipboard
| Challenge: | acrostics are hidden messages in which initial letters of consecutive lines or paragraphs form meaningful words or phrases. |
| Approach: | They propose a method to identify acrostics automatically and rank them by the probability that the corresponding sequence of characters does not occur by chance. |
| Outcome: | The proposed method achieves F1 scores of 0.39, 0.59, and 0.66 on the French, English, and Russian subdomains of WikiSource. |
Semi-Automated Construction of Sense-Annotated Datasets for Practically Any Language (2025.coling-main)
Copied to clipboard
Jai Riley, Bradley M. Hauer, Nafisa Sadaf Hriti, Guoqing Luo, Amir Reza Mirzaei, Ali Rafiei, Hadi Sheikhi, Mahvash Siavashpour, Mohammad Tavakoli, Ning Shi, Grzegorz Kondrak
| Challenge: | Word sense disambiguation is a widely studied NLP task of identifying the meaning of a word in context. |
| Approach: | They propose a method to create parallel sense-annotated datasets in English . they use machine translation, word alignment, sense projection, and sense filtering to produce silver annotations . |
| Outcome: | The proposed method produces parallel sense-annotated datasets on Farsi, Chinese, and Bengali . the results are higher than those obtained with recent multilingual systems, the authors say . |
Complex and Precise Movie and Book Annotations in French Language for Aspect Based Sentiment Analysis (L18-1)
Copied to clipboard
| Challenge: | Aspect Based Sentiment Analysis (ABSA) aims at collecting detailed opinion information according to products and their features. |
| Approach: | They propose to use linguistics tools to enhance text classification with aspect-based sentiment analysis. |
| Outcome: | The proposed method is based on two French online reviews datasets. |
WARM: A Weakly (+Semi) Supervised Math Word Problem Solver (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches to solving math word problems require full supervision in the form of intermediate equations. |
| Approach: | They propose a weakly supervised model that requires only the final answer as supervision to solve math word problems. |
| Outcome: | The proposed model achieves accuracy gains of 4.5% and 32% over current weakly-supervised methods on standard Math23K and AllArith datasets. |
Text Embedding Inversion Security for Multilingual Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | storing sensitive information as embeddings is susceptible to security breaches, as text can be reconstructed from embeddables . study explores multilingual inversion attacks using a masking defense . |
| Approach: | They propose a simple masking defense that can be used to decode embedded text . they define the problem of black-box multilingual and crosslingual inversion attacks . |
| Outcome: | The proposed defense is effective for both monolingual and multilingual models. |
Neural Legal Judgment Prediction in English (P19-1)
Copied to clipboard
| Challenge: | Recent work on legal judgment prediction has focused on Chinese, but only feature-based models have been considered in English. |
| Approach: | They propose a hierarchical version of BERT which bypasses BERT’s length limitation. |
| Outcome: | The proposed model outperforms existing models in binary violation classification, multi-label classification and case importance prediction. |
Distinguishing Address vs. Reference Mentions of Personal Names in Text (2023.findings-acl)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a core task in the NLP community . but not much work has been done to distinguish between addressing and referring to entities . |
| Approach: | They propose an automatic tagger that captures the address vs. reference distinction in English . they demonstrate how this distinction is important in NLP and computational social science applications . |
| Outcome: | The proposed tagger performs at 85% accuracy in distinguishing between address and reference in English . many modern Indo-European languages do not have such vocative case markers . |
Universal Semantic Tagging for English and Mandarin Chinese (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to generating semantic annotations for different languages are attracting more and more interest. |
| Approach: | They propose to extend Universal Semantic Tagging to Mandarin Chinese and evaluate its performance. |
| Outcome: | The proposed scheme is only tested in four Indo–European languages . accuracies are 92.7% and 94.6% for Chinese and English respectively . |
Neural Machine Translation for Low-Resourced Indian Languages (2020.lrec-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is an effective way to convert text to a different language without human involvement. |
| Approach: | They propose to use multihead self-attention along with pre-trained Byte-Pair-Encoded (BPE) and MultiBPE embeddings to develop an efficient machine translation system. |
| Outcome: | The proposed system outperforms Google translator and the existing translators on two of the most morphological rich Indian languages. |
GreekMMLU: A Native-Sourced Multitask Benchmark for Evaluating Language Models in Greek (2026.findings-acl)
Copied to clipboard
Yang Zhang, Mersin Konomi, Christos Xypolopoulos, Konstantinos Divriotis, Konstantinos Skianis, Giannis Nikolentzos, Giorgos Stamou, Guokan Shang, Michalis Vazirgiannis
| Challenge: | Existing evaluation benchmarks for large language models are limited for Greek . Existing datasets are often machine-translated from English, failing to capture Greek linguistic and cultural characteristics. |
| Approach: | They propose a native-sourced benchmark for massive multitask language understanding in Greek . they publicize 16,857 samples and reserve 4,948 samples for a private leaderboard . |
| Outcome: | The proposed model is based on 21,805 multiple-choice questions across 45 subject areas . the model is publicly released and reserved for a private leaderboard . |
A Post-Editing Dataset in the Legal Domain: Do we Underestimate Neural Machine Translation Quality? (2020.lrec-1)
Copied to clipboard
Julia Ive, Lucia Specia, Sara Szoc, Tom Vanallemeersch, Joachim Van den Bogaert, Eduardo Farah, Christine Maroti, Artur Ventura, Maxim Khalilov
| Challenge: | Current state-of-the-art in Neural Machine Translation (NMT) has reached remarkable progress, but human evaluations are often judged as having lower quality than top NMT systems. |
| Approach: | They propose to use a machine translation dataset with post-edited high-quality neural machine translation and independent human references to compare the results. |
| Outcome: | The proposed dataset includes 31K tuples including a source sentence, the respective machine translation by a neural machine translation system, and a post-edited version of such translation by professional translator. |
TDDC: Timely Disclosure Documents Corpus (2020.lrec-1)
Copied to clipboard
| Challenge: | TDDC was prepared by manually aligning the sentences from past Japanese and English timely disclosure documents . tens of thousands of original Japanese documents are disclosed every year, but the availability of English disclosure documents is limited. |
| Approach: | They describe the details of the Timely Disclosure Documents Corpus (TDDC) TDDC was prepared by manually aligning the sentences from past Japanese and English timely disclosure documents . |
| Outcome: | The timely disclosure documents corpus (TDDC) was created by aligning sentences from past documents in Japanese and English. |
An Empirical Study of Generation Order for Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | a recent study of generation order for machine translation shows it does not affect output quality . Neural sequence models have been successfully applied to a broad range of tasks in recent years . |
| Approach: | They propose a soft order-reward framework that enables models to follow arbitrary oracle generation policies. |
| Outcome: | The proposed framework explores a wide variety of generation orders including uninformed orders, location-based orders, frequency-based or model-based orderings, and model-driven orders. |
Language Model Quality Correlates with Psychometric Predictive Power in Multiple Languages (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have found that higher quality language models provide more powerful predictors of human reading behavior, but empirical support for the QP hypothesis is mixed. |
| Approach: | They propose to test the quality–power hypothesis by using surprisal language models to test their ability to predict eye tracking data. |
| Outcome: | The proposed model is based on a set of language models with a 'quality-power' hypothesis. |
X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Language models (LMs) capture factual knowledge by filling in the blanks of cloze-style prompts. |
| Approach: | They propose a code-switching-based method to improve the ability of multilingual LMs to access knowledge and verify its effectiveness on several benchmark languages. |
| Outcome: | The proposed method improves the ability of multilingual LMs to access knowledge and verify its effectiveness on several benchmark languages. |
KINNEWS and KIRNEWS: Benchmarking Cross-Lingual Text Classification for Kinyarwanda and Kirundi (2020.coling-main)
Copied to clipboard
| Challenge: | low-resource African languages are traditionally left behind because of the lack of well-annotated data and effective preprocessing. |
| Approach: | They propose two news datasets for multi-class classification of news articles in two low-resource African languages. |
| Outcome: | The proposed datasets show that training embeddings on the higher-resourced Kinyarwanda yields successful cross-lingual transfer to Kirundi. |
CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairs (2020.emnlp-main)
Copied to clipboard
| Challenge: | Cross-lingual document alignment aims to identify pairs of documents in two distinct languages that are of comparable content or translations of each other. |
| Approach: | They exploit the signals embedded in URLs to label web documents at scale with an average precision of 94.5% across different language pairs. |
| Outcome: | The proposed method can label documents at 94.5% across languages with high precision . the proposed method is useful for low-resource languages with limited resources . |
Multilingual Clustering of Streaming News (D18-1)
Copied to clipboard
| Challenge: | a novel method for clustering news across languages is proposed . a key challenge in handling news streams is that they must be generated on the fly . |
| Approach: | They propose a method for clustering news across languages into monolingual and crosslingual clusters . they use real news datasets in multiple languages to find an ever growing number of cluster labels . |
| Outcome: | The proposed method produces state-of-the-art results on real news datasets in German, English and Spanish. |
Morfessor EM+Prune: Improved Subword Segmentation with Expectation Maximization and Pruning (2020.lrec-1)
Copied to clipboard
| Challenge: | Subword segmentation is a standard preprocessing step in many neural approaches to natural language processing. |
| Approach: | They propose to train a unigram subword model using a recursive algorithm and lexicon pruning algorithm. |
| Outcome: | The proposed method improves on the original training algorithm and improves morphological segmentation accuracy. |
Finding Universal Grammatical Relations in Multilingual BERT (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work has found that multilingual masked language models learn a surprising amount of linguistic structure, despite a lack of direct linguistic supervision. |
| Approach: | They propose an unsupervised method to find syntactic tree distances in languages other than English and that these subspaces are approximately shared across languages. |
| Outcome: | The proposed method shows that mBERT learns representations of syntactic dependency labels, in the form of clusters, which largely agree with the Universal Dependencies taxonomy. |
Multilingual Culture-Independent Word Analogy Datasets (2020.lrec-1)
Copied to clipboard
| Challenge: | In text processing, deep neural networks use word embeddings as an input. |
| Approach: | They propose to use benchmark datasets to compare the quality of word embeddings in text processing . they use a word analogy task in Croatian, English, Estonian, Finnish, Latvian, Lithuanian, Russian, Slovenian, and Swedish . |
| Outcome: | The proposed datasets are culturally independent and cross-lingual for the languages used. |
UniSent: Universal Adaptable Sentiment Lexica for 1000+ Languages (2020.lrec-1)
Copied to clipboard
| Challenge: | Sentiment lexica are vital for sentiment analysis in absence of document-level annotations . linguistic resources are limited for at least a few hundred languages, putting them at risk of extinction . |
| Approach: | They introduce UniSent universal sentiment lexica for 1000+ languages . they use a Bible corpus to project sentiment information from English to other languages based on Twitter data . |
| Outcome: | The proposed method mitigates domain mismatch between Bible and Twitter by using embeddings . it compares to other sentiment seeding methods in a subset of languages with ground truth available . |
Chinese Paragraph-level Discourse Parsing with Global Backward and Local Reverse Reading (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods on discourse parsing in English suffer from long discourse units and fewer explicit connectives. |
| Approach: | They propose to use two reading modes to construct Chinese paragraph level discourse trees. |
| Outcome: | The proposed model outperforms baselines on Chinese discourse trees. |
MILU: A Multi-task Indic Language Understanding Benchmark (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing benchmarks focus on English, leaving substantial gaps in assessing LLM capabilities in low-resource and linguistically diverse languages. |
| Approach: | They propose a multi-task indic language understanding benchmark to assess LLMs in low-resource languages. |
| Outcome: | The new benchmark spans 8 domains and 41 subjects across 11 Indic languages, reflecting general and culturally specific knowledge. |
Qorǵau: Evaluating Safety in Kazakh-Russian Bilingual Contexts (2025.findings-acl)
Copied to clipboard
Maiya Goloburda, Nurkhan Laiyk, Diana Turmakhan, Yuxia Wang, Mukhammed Togmanov, Jonibek Mansurov, Askhat Sametov, Nurdaulet Mukhituly, Minghan Wang, Daniil Orel, Zain Muhammad Mujahid, Fajri Koto, Timothy Baldwin, Preslav Nakov
| Challenge: | Large language models (LLMs) have the potential to generate harmful content, posing risks to users. |
| Approach: | They propose a dataset specifically designed for safety evaluation in Kazakh and Russian . they use a bilingual context in Kazakhstan where both Kazakh (a low-resource language) and Russian (a high-resourced language) |
| Outcome: | The proposed dataset is designed for safety evaluation in Kazakh and Russian . it shows that both multilingual and language-specific LLMs perform better than others . |
Leveraging Large Pre-trained Multilingual Models for High-Quality Speech-to-Text Translation on Industry Scenarios (2025.coling-main)
Copied to clipboard
| Challenge: | Speech-to-Text Translation systems rely on a sequential pipeline that combines ASR and MT models. |
| Approach: | They propose a parameter-efficient framework that integrates one LPSM with a multilingual MT engine. |
| Outcome: | The proposed framework integrates one LPSM with a multilingual MT engine. |
Adversarial Training for Low-Resource Disfluency Correction (2023.findings-acl)
Copied to clipboard
| Challenge: | Disfluencies can be introduced in conversational speech due to the conversational nature of speech and/or speech impairments such as stuttering. |
| Approach: | They propose an adversarial sequence-tagging model for Disfluency Correction . they evaluate it in Bengali, Hindi, and Marathi languages and use it to correct stuttering disfluencies . |
| Outcome: | The proposed technique improves in Bengali, Hindi, and Marathi languages . it also removes stuttering disfluencies in ASR transcripts introduced by speech impairments . |
Learn and Unlearn: Addressing Misinformation in Multilingual LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to unlearning large language models (LLMs) focus on English data, but they ignore multilingual contexts and can produce misleading, offensive, or otherwise fake content. |
| Approach: | They investigate the propagation of information in multilingual large language models and evaluate unlearning methods to address harmful content in multi-lingual contexts. |
| Outcome: | The proposed methods can effectively eliminate harmful content for all languages by addressing both English and the original language of the harmful data. |
EMPEC: A Comprehensive Benchmark for Evaluating Large Language Models Across Diverse Healthcare Professions (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) show their potential in accurately answering biomedical questions, yet current healthcare benchmarks primarily assess knowledge mastered by medical doctors, neglecting other essential professions. |
| Approach: | They evaluated 17 LLMs including proprietary and open-source models and found they struggled with specialized fields and alternative medicine. |
| Outcome: | The examinations for medical PErsonnel in Chinese (EMPEC) features 157,803 exam questions across 124 subjects and 20 healthcare professions. |
A Simple and Effective Approach to Robust Unsupervised Bilingual Dictionary Induction (2020.coling-main)
Copied to clipboard
| Challenge: | Recent work has questioned the robustness of unsupervised bilingual dictionary induction methods on distant language pairs. |
| Approach: | They propose an iterative dimension reduction method to bridge this gap . they propose a method that initializes and self-learning and inducing a dictionary . |
| Outcome: | The proposed method achieves 13.64 55.53% accuracy between English and four distant languages. |
Improving Machine Translation of Educational Content via Crowdsourcing (L18-1)
Copied to clipboard
Maximiliana Behnke, Antonio Valerio Miceli Barone, Rico Sennrich, Vilelmini Sosoni, Thanasis Naskos, Eirini Takoulidou, Maria Stasimioti, Menno van Zaanen, Sheila Castilho, Federico Gaspari, Panayota Georgakopoulou, Valia Kordoni, Markus Egg, Katia Lida Kermanidis
| Challenge: | Using crowdsourcing to train neural machine translation models is expensive and expensive . professional outsourcing of bilingual data is expensive if the translations are of a lower quality . |
| Approach: | They analyze the impact of crowdsourcing on the quality of in-domain training data . they use translations of MOOCs from English to eleven languages to fine-tune machine translation models . |
| Outcome: | The proposed method improves on general-domain training data and with pre-existing in-domain corpora. |
Is the Brain Mechanism for Hierarchical Structure Building Universal Across Languages? An fMRI Study of Chinese and English (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that the brain builds hierarchical syntactic structures, but it is unknown whether they are universal across languages. |
| Approach: | They analyze the working memory requirements when applying parsing strategies to two languages: Chinese and English. |
| Outcome: | The proposed method shows that the brain adopts parsing strategies with less memory load according to different language structures. |
A Large Parallel Corpus of Full-Text Scientific Articles (L18-1)
Copied to clipboard
| Challenge: | Scielo database contains articles from several research domains. |
| Approach: | They propose to build a parallel corpus from Scielo in three languages: English, Portuguese, and Spanish. |
| Outcome: | The proposed system outperforms other systems on scientific articles in English, Portuguese, and Spanish. |
Extracting an English-Persian Parallel Corpus from Comparable Corpora (L18-1)
Copied to clipboard
| Challenge: | Existing methods to extract parallel sentences from Wikipedia are limited for some language pairs such as Persian-English. |
| Approach: | They propose a bidirectional method to extract parallel sentences from Wikipedia . they add extracted sentences to existing training data and use IR system to measure similarity . |
| Outcome: | The proposed method outperforms the one-directional approach in analyzing translation data from two translation systems and IR systems. |
Error Analysis of NLP Models and Non-Native Speakers of English Identifying Sarcasm in Reddit Comments (2024.lrec-main)
Copied to clipboard
| Challenge: | sarcasm detection remains an issue for both humans and natural language processing models . |
| Approach: | They analysed 300 comments from the FigLang 2020 Reddit Dataset and 39 non-native speakers of English to see if they were sarcastic. |
| Outcome: | The results show that the models and models have similar performance and weaknesses when the comments include political topics or are phrased as questions. |
Building Named Entity Recognition Taggers via Parallel Corpora (L18-1)
Copied to clipboard
| Challenge: | Existing methods to generate semantic processors for languages lacking hand curated data are inefficiently slow and unaffordable in terms of human resources and economic costs. |
| Approach: | They propose to use statistical word alignments to project annotations from multiple sources to a target language. |
| Outcome: | The proposed method is effective to transport NER annotations across languages . it can generate a good statistical model for a new target language . |
Evaluating Shortest Edit Script Methods for Contextual Lemmatization (2024.lrec-main)
Copied to clipboard
| Challenge: | Modern contextual lemmatizers often rely on automatically induced Shortest Edit Scripts (SES) supervised contextual methods are used to perform lemma classification tasks. |
| Approach: | They propose to use masked language encoders to compute shortest edit Scripts (SES) SES is the number of edit operations to transform a word form into its lemma . |
| Outcome: | The proposed model outperforms language-specific models in all evaluation settings with seven languages of different morphological complexity. |
Multilingual Event Extraction from Historical Newspaper Adverts (2023.acl-long)
Copied to clipboard
| Challenge: | Developing NLP methods for historical corpora is difficult, as only domain experts can label them . off-the-shelf models are trained on modern language texts, rendering them weaker for historical documents . |
| Approach: | They propose to use an annotated newspaper dataset to extract historical data from a novel domain of texts. |
| Outcome: | The proposed method performs well on a multilingual dataset in English, French, and Dutch . it is possible to extract surprisingly good results even with scarce annotated data using existing models and datasets for modern languages . |
SEAHORSE: A Multilingual, Multifaceted Dataset for Summarization Evaluation (2023.emnlp-main)
Copied to clipboard
Elizabeth Clark, Shruti Rijhwani, Sebastian Gehrmann, Joshua Maynez, Roee Aharoni, Vitaly Nikolaev, Thibault Sellam, Aditya Siddhant, Dipanjan Das, Ankur Parikh
| Challenge: | evaluating the quality of generated text is a difficult problem for large language models. |
| Approach: | They propose a dataset for multilingual, multifaceted summarization evaluation. |
| Outcome: | The proposed dataset can be used to train multilingual summarization systems . it shows that the dataset performs well on the out-of-domain meta-evaluation benchmarks TRUE and mFACE . |
Recovering document annotations for sentence-level bitext (2024.findings-acl)
Copied to clipboard
| Challenge: | In machine translation, historical models were incapable of handling longer contexts, so the lack of document-level datasets was less noticeable. |
| Approach: | They propose a document-level filtering technique that discards document- level metadata. |
| Outcome: | The proposed method improves translation without degradation of sentence-level translation. |
Chat Vector: A Simple Approach to Equip LLMs with Instruction Following and Model Alignment in New Languages (2024.acl-long)
Copied to clipboard
Shih-Cheng Huang, Pin-Zu Li, Yu-chi Hsu, Kuang-Ming Chen, Yu Tung Lin, Shih-Kai Hsiao, Richard Tsai, Hung-yi Lee
| Challenge: | Despite the rapid development of large language models, the language capabilities of most open-source LLMs are primarily focused on English due to data constraints. |
| Approach: | They propose a chat vector to equip pre-trained language models with instruction following and human value alignment via simple model arithmetic. |
| Outcome: | The proposed method can be extended to include various languages, base models, and chat vectors. |
On the Similarity of Circuits across Languages: a Case Study on the Subject-verb Agreement Task (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Several algorithms implemented by language models have been successfully reverse-engineered, leaving it unclear how universal circuits are across different settings. |
| Approach: | They propose to use Gemma 2B to solve the subject-verb agreement task across two different languages, English and Spanish. |
| Outcome: | The proposed circuits solve the subject-verb agreement task across two different languages, and are language-independent and language-dependent. |
Extending LLMs to New Languages: A Case Study of Llama and Persian Adaptation (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are mainly trained on English data and struggle with low-resource languages. |
| Approach: | They propose to add a new language to Llama to improve classification accuracy for Persian tasks by aligning representations through bilingual pretraining and instruction datasets. |
| Outcome: | The proposed model performs on generation and classification tasks with no adverse impact and sometimes even improvements on English tasks. |
IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages (2024.acl-long)
Copied to clipboard
| Challenge: | IndicGenBench is the largest benchmark for evaluating large language models on user-facing generation tasks across a diverse set of 29 Indic languages . |
| Approach: | They evaluate large language models on user-facing generation tasks across 29 languages . they use human curation to provide multi-way parallel evaluation data for many under-represented languages a github repository . |
| Outcome: | IndicGenBench is the largest benchmark for evaluating LLMs on user-facing generation tasks across a diverse set of 29 Indic languages covering 13 scripts and 4 language families. |
Selected Languages are All You Need for Cross-lingual Truthfulness Transfer (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for truthfulness enhancement in English are limited to multilingual scenarios. |
| Approach: | They propose a method for cross-lingual truthfulness transfer that uses language bias and transfer contributions to select an optimal subset of all tested languages and employ translation instruction tuning for cross language truthfulness transfers. |
| Outcome: | The proposed method reduces multilingual representation disparity and boosts cross-lingual truthfulness transfer of LLMs. |
TeSum: Human-Generated Abstractive Summarization Corpus for Telugu (2022.lrec-1)
Copied to clipboard
| Challenge: | a number of recent datasets for summarisation, scraped the web-content relying on the assumption that summary is made available with the article by the publishers. |
| Approach: | They propose a pipeline that crowd-sources summarization data and then aggressively filters the content via: automatic and partial expert evaluation. |
| Outcome: | The proposed pipeline can be applied to scraped datasets to extract better quality articles-summaries pairs. |
WordNet Is All You Need: A Surprisingly Effective Unsupervised Method for Graded Lexical Entailment (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a simple unsupervised method for predicting graded lexical entailment in English relies on WordNet . despite its simplicity, our method outperforms all previous methods using WordNet as weak supervision. |
| Approach: | They propose an unsupervised method which relies exclusively on WordNet for predicting graded lexical entailment in English. |
| Outcome: | The proposed method outperforms existing methods on the largest GLE dataset using WordNet. |
MULTITuDE: Large-Scale Multilingual Machine-Generated Text Detection Benchmark (2023.emnlp-main)
Copied to clipboard
Dominik Macko, Robert Moro, Adaku Uchendu, Jason Lucas, Michiharu Yamashita, Matúš Pikuliak, Ivan Srba, Thai Le, Dongwon Lee, Jakub Simko, Maria Bielikova
| Challenge: | MULTITuDE benchmarks lack authentic and machine-generated text in languages other than English . defining characteristic of new generation of LLMs is increased quality of text . |
| Approach: | They propose a benchmarking dataset for multilingual machine-generated text detection that compares detectors with authentic and machine-generated texts in 11 languages. |
| Outcome: | The proposed dataset compares detectors with zero-shot and fine-tuned detectors in 11 languages. |
Few-shot Learning with Multilingual Generative Language Models (2022.emnlp-main)
Copied to clipboard
Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O’Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, Veselin Stoyanov, Xian Li
| Challenge: | Large-scale generative language models such as GPT-3 are competitive few-shot learners. |
| Approach: | They train multilingual generative language models on a corpus covering a diverse set of languages and study their few- and zero-shot learning capabilities. |
| Outcome: | The proposed model outperforms GPT-3 on 171 out of 182 directions with 32 training examples and surpasses the official supervised baseline in 45 directions. |
FrameEOL: Semantic Frame Induction using Causal Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Semantic frame induction is the task of clustering frame-evoking words according to the semantic frames they evoke. |
| Approach: | They propose a prompt-based method for obtaining Frame Embeddings that outputs One frame-name as a Label . |
| Outcome: | The proposed method outperforms existing methods on English and Japanese datasets. |
FalAI: A Dataset for End-to-end Spoken Language Understanding in a Low-Resource Scenario (2024.lrec-main)
Copied to clipboard
Andres Pineiro-Martin, Carmen Garcia-Mateo, Laura Docio-Fernandez, Maria del Carmen Lopez-Perez, Jose Gandarela-Rodriguez
| Challenge: | End-to-end (E2E) Spoken Language Understanding systems extract structured information from speech signals using a single model. |
| Approach: | They propose to use a dataset to extract structured information from speech signals . they define splits for noisy audio, hesitant audio and audio where sentence has changed . |
| Outcome: | The proposed model exploits acoustic information and avoids cascading errors . falAI dataset is the largest public SLU dataset in Galician and first to be obtained in low-resource scenario. |
Offensive Language Identification in Greek (2020.lrec-1)
Copied to clipboard
| Challenge: | a gap in the literature on offensive language has been addressed with studies on Spanish, Hindi, and German. |
| Approach: | They present a Greek annotated dataset for offensive language identification . it contains 4,779 tweets annotating offensive and not offensive posts from Twitter . they evaluate several computational models trained and tested on the dataset . |
| Outcome: | The proposed dataset contains 4,779 tweets annotated as offensive and not offensive . the authors show that the proposed dataset is similar to the OLID dataset for English . |
Detecting Sensitive Personal Information in Japanese Pre-Training Corpora for Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large-scale pre-training corpora are essential for large language models, but if such content remains unfiltered, there is a risk that LLMs may memorize it and leak it through their outputs. |
| Approach: | They construct a Japanese text corpora dataset and train machine learning models to detect SCPI in text. |
| Outcome: | The proposed classifier can detect information related to SCPI in Japanese text. |
FRACAS: a FRench Annotated Corpus of Attribution relations in newS (2024.lrec-main)
Copied to clipboard
| Challenge: | Quotation extraction is a useful task, but it is not widely studied in other languages. |
| Approach: | They propose to annotate a manually annotated corpus of 1,676 newswire texts in French for quotation extraction and source attribution. |
| Outcome: | The proposed system is compared to the most recent system for quotation extraction in the French language. |
Code-Switching Red-Teaming: LLM Evaluation for Safety and Multilingual Understanding (2025.acl-long)
Copied to clipboard
| Challenge: | Recent large language models (LLMs) are inherently multilingual agents . concerns regarding their safety have emerged . |
| Approach: | They propose a framework to synthesize red-teaming queries and investigate their safety . they demonstrate that the framework outperforms existing red- teaming techniques . |
| Outcome: | The proposed framework outperforms existing red-teaming techniques in the safety domain . it generates code-switching attack prompts in monolingual data . |
DREsS: Dataset for Rubric-based Essay Scoring on EFL Writing (2025.acl-long)
Copied to clipboard
| Challenge: | Automated essay scoring (AES) is a useful tool in English as a foreign language (EFL) writing education. |
| Approach: | They propose a large-scale, standard dataset for rubric-based automated essay scoring with 48.9K samples in total. |
| Outcome: | The proposed system improves the baseline scores by 45.44%. |
Question-Answering in a Low-resourced Language: Benchmark Dataset and Models for Tigrinya (2023.acl-long)
Copied to clipboard
| Challenge: | Question-Answering (QA) has seen significant advances in recent years, achieving near human-level performance over some benchmarks. |
| Approach: | They propose to use a native QA dataset for an East African language, Tigrinya, to build similar resources for related languages. |
| Outcome: | The proposed method is applicable to constructing similar resources for related languages. |
SenSALDO: Creating a Sentiment Lexicon for Swedish (L18-1)
Copied to clipboard
| Challenge: | sentiment analysis has seen an explosive expansion over the last decade or so . many theoretical and methodological questions remain unanswered and resource gaps unfilled . |
| Approach: | They develop a sentiment lexicon for written (standard) Swedish using an existing dataset . they assign a real value sentiment score in the range [-1,1] and produce a label for it . |
| Outcome: | The proposed sentiment lexicon is an open source resource from the Swedish Language Bank . it is based on an existing gold standard dataset and is available from Sprkbanken . |
Assessing Users’ Reputation from Syntactic and Semantic Information in Community Question Answering (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing reputation systems do not take linguistic quality into account in reputation scores estimation. |
| Approach: | They build statistical models that learn reputation from syntactic and semantic structures extracted from their associated answers content. |
| Outcome: | The proposed models show that users’ writing styles play important roles in building reputation points. |
Cross-Lingual Cross-Target Stance Detection with Dual Knowledge Distillation Framework (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on stance detection were conducted mainly in English due to the low-resource problem in most non-English languages. |
| Approach: | They propose to use a cross-lingual teacher and a teacher to transfer knowledge from source to target language to bridge the discrepancy between languages. |
| Outcome: | The proposed framework bridges the discrepancy between languages and generalizes the knowledge to unseen targets in target language. |
German Also Hallucinates! Inconsistency Detection in News Summaries with the Absinth Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have made significant progress on a wide range of natural language processing tasks, but they still suffer from hallucinating information in their output. |
| Approach: | They propose to use an annotated dataset to detect hallucinations in german news summarization and open-source it to foster further research on hallucinosity detection in german. |
| Outcome: | The proposed model can detect hallucinations in the output and evaluate the faithfulness of the summaries. |
Peacock: A Family of Arabic Multimodal Large Language Models and Benchmarks (2024.acl-long)
Copied to clipboard
Fakhraddin Alwajih, El Moatez Billah Nagoudi, Gagan Bhatia, Abdelrahman Mohamed, Muhammad Abdul-Mageed
| Challenge: | MLLMs have proven effective in a wide range of tasks that require complex reasoning and linguistic comprehension, but they are limited to English-based settings. |
| Approach: | They propose a family of Arabic multimodal large language models with strong vision and language capabilities. |
| Outcome: | The proposed models show strong performance on visual reasoning tasks and language capabilities. |
Efficient Entity Candidate Generation for Low-Resource Languages (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing approaches for cross-lingual entity linking are not suitable for English. |
| Approach: | They propose a candidate generation problem in cross-lingual entity linking with a focus on low-resource languages. |
| Outcome: | The proposed solution outperforms the state-of-the-art approach on 9 real-world datasets and query types. |
GPT-3.5 for Grammatical Error Correction (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent work shows that GPT-3.5 struggles with several error types, including punctuation mistakes, tense errors, syntactic dependencies between words, and lexical compatibility at the sentence level. |
| Approach: | They evaluate GPT-3.5 for grammatical error correction in multiple languages . they use it to re-rank correction hypotheses generated by other GEC models . |
| Outcome: | The proposed model performs well in English and Russian, but struggles with errors in other languages. |
Improving the Detection of Multilingual Online Attacks with Rich Social Media Data from Singapore (2023.acl-long)
Copied to clipboard
Janosch Haber, Bertie Vidgen, Matthew Chapman, Vibhor Agarwal, Roy Ka-Wei Lee, Yong Keong Yap, Paul Röttger
| Challenge: | Toxic content is a global problem, but most resources for detecting toxic content are in English . new datasets and models for non-English languages focus exclusively on one language or dialect . |
| Approach: | They propose to use a multilingual dataset of online attacks to identify code-mixed toxic content in Singapore . they collect reddit comments in Indonesian, Malay, Singlish, and other languages and provide fine-grained hierarchical labels for attacks . |
| Outcome: | The proposed dataset provides fine-grained hierarchical labels for online attacks in Singapore . it shows that the metadata can be used for granular error analysis . |
SwiLTra-Bench: The Swiss Legal Translation Benchmark (2025.acl-long)
Copied to clipboard
Joel Niklaus, Jakob Merane, Luka Nenadic, Sina Ahmadi, Yingqiang Gao, Cyrill A. H. Chevalley, Claude Humbel, Christophe Gösken, Lorenzo Tanzi, Thomas Lüthi, Stefan Palombo, Spencer Poff, Boling Yang, Nan Wu, Matthew Guillod, Robin Mamié, Daniel Brunner, Julio Pereyra, Niko Grupen
| Challenge: | In Switzerland legal translation relies on legal experts who must be both legal experts and skilled translators—creating bottlenecks and impacting effective access to justice. |
| Approach: | They propose a multilingual benchmarking system that evaluates Swiss legal translation systems based on 180K aligned Swiss legal translator pairs . they show frontier models achieve superior translation performance across all document types while specialized translation systems excel specifically in laws but under-perform in headnotes. |
| Outcome: | The proposed model outperforms specialized models in laws but underperform in headnotes. |
Evaluating Robustness of Large Language Models Against Multilingual Typographical Errors (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly deployed in multilingual, real-world applications where user inputs introduce typographical errors. |
| Approach: | They propose a multilingual typo generation algorithm that simulates human-like errors based on language-specific keyboard layouts and typing behavior. |
| Outcome: | The proposed model can generate the correct answer ("500") under typos in English, German, and Russian. |
RLHF Can Speak Many Languages: Unlocking Multilingual Preference Optimization for LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Preference optimization is a widely adopted post-training technique to align large language models with human preferences. |
| Approach: | They propose a method for generating multilingual feedback data to balance data coverage. |
| Outcome: | The proposed method achieves 54.4% win-rate against current state-of-the-art multilingual LLM in its parameter class and 69.5% win- rate or higher against widely used models like Gemma, Mistral and Llama 3. |
COGS: A Compositional Generalization Challenge Based on Semantic Interpretation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Natural language is characterized by compositionality: meaning of complex expressions is constructed from the meanings of its constituent parts. |
| Approach: | They propose a semantic parsing dataset based on a fragment of English to assess compositional generalization abilities. |
| Outcome: | The proposed model can generalize meanings in a given sentence in 96–99% of the tests, but generalization accuracy is lower and the generalization sensitivity is higher. |
BARThez: a Skilled Pretrained French Sequence-to-Sequence Model (2021.emnlp-main)
Copied to clipboard
| Challenge: | Inductive transfer learning has taken the entire NLU field by storm, with models such as BERT and BART setting new state-of-the-art on countless tasks. |
| Approach: | They introduce a large-scale pretrained seq2seq model for French that is very competitive with state-of-the-art BERT-based French language models such as CamemBERT and FlauBERT. |
| Outcome: | The proposed model outperforms existing models on discriminative and generative tasks on a French summarization dataset. |
KOLD: Korean Offensive Language Dataset (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent directions for offensive language detection focus on English and do not transfer well to other languages because of cultural and linguistic differences. |
| Approach: | They present a Korean offensive language dataset annotated with offensive language comments . they use the comments as training data for Korean BERT and RoBERTa models . |
| Outcome: | The proposed model improves offensiveness detection, target classification, and span detection while having room for improvement for target group classification and span prediction. |
Cross-lingual Emotion Detection (2022.lrec-1)
Copied to clipboard
| Challenge: | Emotion detection is a useful tool for understanding human behavior, but constructing annotated datasets to train models can be expensive. |
| Approach: | They propose to use English as the source language with Arabic and Spanish as target languages to train models for emotion detection in a target language. |
| Outcome: | The proposed approaches surpass state-of-the-art models in Arabic and Spanish by 4% and 5% respectively. |
BiMediX2 : Bio-Medical EXpert LMM for Diverse Medical Modalities (2025.findings-emnlp)
Copied to clipboard
Sahal Shaji Mullappilly, Mohammed Irfan Kurpath, Sara Pieri, Saeed Yahya Alseiari, Shanavas Cholakkal, Khaled M Aldahmani, Fahad Shahbaz Khan, Rao Muhammad Anwer, Salman Khan, Timothy Baldwin, Hisham Cholakkal
| Challenge: | BiMediX2 is a bilingual (Arabic-English) large multimodal model that supports text-based and image-based medical interactions. |
| Approach: | They introduce BiMediX2, a bilingual (Arabic-English) Bio-Medical EXpert Large Multimodal Model that supports text-based and image-based medical interactions. |
| Outcome: | The model outperforms existing models by over 9% in English and more than 20% in Arabic evaluations. |
CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies on social biases in language models have focused on only English. |
| Approach: | They propose to use a Chinese dataset for bias evaluation and mitigation of Chinese conversational language models. |
| Outcome: | The proposed dataset includes under-explored bias categories, such as ageism and appearance biases, which received less attention in previous studies. |
Detecting Troll Tweets in a Bilingual Corpus (2020.lrec-1)
Copied to clipboard
| Challenge: | a large amount of troll accounts have emerged with efforts to manipulate public opinion on social network sites . a recent study found that trolled tweets spread misinformation, fake news, and propaganda . we use supervised classification to detect trol tweets in both English and Russian . |
| Approach: | They propose to detect troll tweets in English and Russian using machine learning algorithms . they use monolingual, cross-lingual, and bilingual training scenarios . |
| Outcome: | The proposed method uses monolingual, cross-lingual, and bilingual training scenarios. |
RoBERTuito: a pre-trained language model for social media text in Spanish (2022.lrec-1)
Copied to clipboard
| Challenge: | Pre-trained language models have been used in many natural language processing tasks . some domain-specific models have shown to improve performance in some domains . however, for languages other than English, such models are not widely available . |
| Approach: | They present a pre-trained language model for user-generated text in Spanish . it is based on 500 million tweets and has some cross-lingual abilities . |
| Outcome: | The model outperforms models trained on over 500 million tweets on a benchmark in spanish and english. |
IndicFinNLP: Financial Natural Language Processing for Indian Languages (2024.lrec-main)
Copied to clipboard
| Challenge: | IndicFinNLP is a collection of 9 datasets relating to FinNLP for three Indian languages. |
| Approach: | They propose to use financial NLP to detect exaggerated numerals in financial texts written in Hindi, Bengali, and Telugu. |
| Outcome: | The proposed framework detects exaggerated numerals in financial texts written in Hindi, Bengali, and Telugu. |
Visually Grounded Reasoning across Languages and Cultures (2021.emnlp-main)
Copied to clipboard
| Challenge: | a new protocol allows for a multilingual hierarchy of concepts and images based on native speakers . the results suggest that the current models are not robust enough to handle multilingual data . |
| Approach: | They propose a protocol to construct an ImageNet-style hierarchy representative of more languages and cultures. |
| Outcome: | The proposed protocol lets the selection of concepts and images be entirely driven by native speakers, rather than scraping them automatically. |
PakBBQ: A Culturally Adapted Bias Benchmark for QA (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are widely adopted in language processing applications, but they often perpetuate harmful societal biases. |
| Approach: | They propose a culturally and regionally adapted extension of the original Bias Benchmark for Question Answering dataset to address this gap. |
| Outcome: | The proposed model gains 12% accuracy with disambiguation and stronger counter bias behaviors in Urdu than in English. |
How Do Multilingual Language Models Remember Facts? (2025.findings-acl)
Copied to clipboard
| Challenge: | Prior research has focused on English monolingual models, but how these mechanisms generalize to non-English languages remains unexplored. |
| Approach: | They analyze three multilingual LLMs to find out how they can generalize recall mechanisms . they find that subject enrichment is language-independent, object extraction is language dependent . |
| Outcome: | The proposed model performs better in multilingual contexts than in English models . the model is more efficient in multi-lingual context, but it is more complex in multilinguistic models compared to English models. |
KazParC: Kazakh Parallel Corpus for Machine Translation (2024.lrec-main)
Copied to clipboard
| Challenge: | Statistical machine translation gained ground over rule-based machine translation in the late 1990s thanks to its ability to learn from large bilingual corpora. |
| Approach: | They propose to develop a parallel corpus for machine translation across Kazakh, English, Russian, and Turkish. |
| Outcome: | The proposed model outperforms Google Translate and Yandex Translate in terms of performance and evaluation metrics. |
EfficientXLang: Towards Improving Token Efficiency Through Cross-Lingual Reasoning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Despite recent advances in Reasoning Language Models, most research focuses solely on English, even though many models are pretrained on multilingual data. |
| Approach: | They evaluate three open-source RLMs: DeepSeek R1, Qwen 2.5, and Qwend 3 across four math datasets and seven typologically diverse languages. |
| Outcome: | The proposed model reduces token usage and preserves accuracy even after translation into English. |
Exploring Large Language Models for Classical Philology (2023.acl-long)
Copied to clipboard
| Challenge: | Recent advances in NLP have led to the creation of powerful language models for many languages including Ancient Greek and Latin. |
| Approach: | They propose to use encoder-only and encoder decoder architectures to create four models for Ancient Greek that vary along two dimensions for tasks of interest for Classical languages. |
| Outcome: | The proposed models improve on existing models of Ancient Greek and Latin and provide a large pre-training corpus for Ancient Greek to support the creation of a larger, comparable model zoo for Classical Philology. |
KIT-19: A Comprehensive Korean Instruction Toolkit on 19 Tasks for Fine-Tuning Korean Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Instruction tuning on large language models is an essential process for models to function well and achieve high performance in the specific tasks. |
| Approach: | They propose to use KIT-19 as an instruction dataset for the development of LLM in Korean to demonstrate its effectiveness. |
| Outcome: | The proposed model outperforms existing Korean LLMs. |
Language Pivoting from Parallel Corpora for Word Sense Disambiguation of Historical Languages: A Case Study on Latin (2024.lrec-main)
Copied to clipboard
| Challenge: | Word Sense Disambiguation (WSD) is an important task in NLP . most of the work on this task has been done on contemporary English or other modern languages, leaving challenges posed by low-resource languages and diachronic change open. |
| Approach: | They propose to use existing bilingual corpora instead of native English datasets to generate a Latin WSD model. |
| Outcome: | The proposed approach achieves state-of-the-art on a standard benchmark for Latin WSD. |
CTAP for Italian: Integrating Components for the Analysis of Italian into a Multilingual Linguistic Complexity Analysis Tool (2020.lrec-1)
Copied to clipboard
| Challenge: | Linguistic complexity is a core construct in Second Language Acquisition (SLA) research. |
| Approach: | They present an open source linguistic complexity measurement tool for Italian . they compare it to existing tools for English and germany . |
| Outcome: | The proposed tool is the most comprehensive linguistic complexity measurement tool for italian . it can be used to compare italian texts to multiple other languages in one tool . |
In the LLM era, Word Sense Induction remains unsolved (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing word sense induction methods rely on predefined sense inventories of arbitrary granularity and require labor intensive sense-annotated data. |
| Approach: | They propose to evaluate an LLM-based word sense induction method for English . they use data augmentation sources, corpus and lexicon, and semi-supervised scenarios to augment the data. |
| Outcome: | The proposed method surpasses the previous SOTA system on a English set by 3.3%. |
Leveraging Contrastive Learning and Knowledge Distillation for Incomplete Modality Rumor Detection (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing rumor detection models neglect the semantic coherence between text and image components in multimodal posts . Existing models neglect incomplete modalities in single modal posts, such as missing text or images . |
| Approach: | They propose a framework for incomplete modality rumor detection that captures semantic consistency between text and image pairs while enhancing model generalization to incomplete modalities within individual posts. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on two English and two Chinese benchmark datasets for rumor detection in social media. |
Multimodal Fusion and Coherence Modeling for Video Topic Segmentation (2025.findings-acl)
Copied to clipboard
| Challenge: | Traditional video topic segmentation methods struggle to discern topical transitions . supervised approaches have improved performance on video action or scene segmentation . |
| Approach: | They propose a new task for video topic segmentation that enhances multimodality alignment and fusion by exploring different architectures using Cross-Attention and Mixture of Experts. |
| Outcome: | The proposed model improves on educational videos, in the form of lectures . it combines cross-attention and mixture of experts to strengthen multimodality alignment and fusion . |
Deep Exploration of Cross-Lingual Zero-Shot Generalization in Instruction Tuning (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on instruction tuning to show cross-lingual generalization . a novel non-English meta-dataset is used to study instruction tuning . |
| Approach: | They perform instruction tuning individually for two distinct language meta-datasets and assess the performance on unseen tasks in a non-English language. |
| Outcome: | The proposed model outperforms baseline training in English and Korean by 20.7% and 13.6%. |
To Ask LLMs about English Grammaticality, Prompt Them in a Different Language (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a study focuses on questions about grammar and fluency in multilingual LLMs . english is the dominant training language for all three models, but prompting in a different language often yields better results. |
| Approach: | They ask three multilingual language models in multiple languages to test their model's grammatical accuracy. |
| Outcome: | The language of the prompt can significantly affect model performance, the study finds . english is the dominant training language for all three models, the researchers show . |
Anecdoctoring: Automated Red-Teaming Across Language and Place (2025.emnlp-main)
Copied to clipboard
| Challenge: | Disinformation is among the top risks of generative AI misuse . red-teaming datasets are typically US- and English-centric . |
| Approach: | They propose a red-teaming approach that generates adversarial prompts across languages and cultures by clustering misinformation claims into broader narratives and enhancing an attacker LLM. |
| Outcome: | The proposed approach produces higher attack success rates and interpretability benefits relative to few-shot prompting. |
MedMT5: An Open-Source Multilingual Text-to-Text LLM for the Medical Domain (2024.lrec-main)
Copied to clipboard
Iker García-Ferrero, Rodrigo Agerri, Aitziber Atutxa Salazar, Elena Cabrio, Iker de la Iglesia, Alberto Lavelli, Bernardo Magnini, Benjamin Molinet, Johana Ramirez-Romero, German Rigau, Jose Maria Villa-Gonzalez, Serena Villata, Andrea Zaninello
| Challenge: | Existing studies on large language models for medical applications have focused on a single language . medical mT5 outperforms both encoders and similar sized text-to-text models in English, French, and Italian benchmarks . |
| Approach: | They propose to train Medical mT5, the first open-source text-to-text multilingual model for the medical domain. |
| Outcome: | The proposed model outperforms encoders and similar sized models on the Spanish, French, and Italian benchmarks while being competitive with current state-of-the-art models in English. |
NoiseBench: Benchmarking the Impact of Real Label Noise on Named Entity Recognition (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to named entity recognition often contain a significant percentage of incorrect labels for entity types and boundary boundaries. |
| Approach: | They propose a noise-robust learning approach that learns from data with partially incorrect labels. |
| Outcome: | The proposed methods are based on simulated noise and are easier to handle than simulated real noise caused by human error or semi-automatic annotation. |
KatFishNet: Detecting LLM-Generated Korean Text through Linguistic Feature Analysis (2025.acl-long)
Copied to clipboard
| Challenge: | Detecting LLM-generated text is crucial for academic integrity, preventing plagiarism, protecting copyrights, ethical research practices. |
| Approach: | They propose a method specifically designed for Korean language to detect LLM-generated text . they examine spacing patterns, part-of-speech diversity, and comma usage . |
| Outcome: | The proposed method achieves an average of 19.78% higher AUC-ROC compared to the best-performing detection method. |
Modal Dependency Parsing via Biaffine Attention with Self-Loop (2025.findings-acl)
Copied to clipboard
| Challenge: | A modal dependency structure is a web of connections between events and sources of information in a document that allows for tracing of who-said-what with what levels of certainty. |
| Approach: | They propose a modal dependency structure that integrates biaffine attention with a large language model to optimize against domain-specific challenges of modal dependence parsing. |
| Outcome: | The proposed approach outperforms the previous state-of-the-art on English and Chinese datasets by 2% and 4% respectively. |
MASSIVE-Agents: A Benchmark for Multilingual Function-Calling in 52 Languages (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Using the original dataset, we cleaned up the MASSIVE dataset and reformatted it for evaluation within the Berkeley Function-Calling Leaderboard framework. |
| Approach: | They present a new benchmark for assessing multilingual function calling across 52 languages . they clean the original MASSIVE dataset and reformat it for evaluation . |
| Outcome: | The new benchmark covers 55 functions and 286 arguments in 52 languages. |
MEMERAG: A Multilingual End-to-End Meta-Evaluation Benchmark for Retrieval Augmented Generation (2025.acl-long)
Copied to clipboard
María Andrea Cruz Blandón, Jayasimha Talur, Bruno Charron, Dong Liu, Saab Mansour, Marcello Federico
| Challenge: | Existing benchmarks focus on English or use translated data, which fails to capture cultural nuances. |
| Approach: | They propose to use a multilingual end-to-end Meta-Evaluation RAG benchmark MEMERAG to assess accuracy and faithfulness of RAG systems. |
| Outcome: | The proposed benchmark can identify improvements offered by advanced prompting techniques and LLMs. |
Towards A “Novel” Benchmark: Evaluating Literary Fiction with Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) context windows have enabled them to process inputs over 100K tokens and generate outputs of up to 10K token. |
| Approach: | They propose a multi-level evaluation framework that incorporates ten metrics across the Macro, Meso, and Micro levels and an annotated fiction dataset. |
| Outcome: | The proposed framework incorporates ten metrics across the Macro, Meso, and Micro levels and is based on a human-human-AI dataset. |
Large Reasoning Models Are (Not Yet) Multilingual Latent Reasoners (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent work shows that large reasoning models arrive at the correct answer before completing textual reasoning steps, indicating the presence of latent reasoning. |
| Approach: | They conduct a systematic investigation of multilingual latent reasoning in large reasoning models across 11 languages. |
| Outcome: | The proposed model arrive at the correct answer before completing the reasoning steps, indicating the presence of latent reasoning. |
Unlocking Multilingual Reasoning Capability of LLMs and LVLMs through Representation Engineering (2026.acl-long)
Copied to clipboard
Qiming Li, Xiaocheng Feng, Yixuan Ma, Ruihan Chen, Zihe Tong, Zekai Ye, Xiachong Feng, Libo Qin, Haoyu Ren, Kun Chen, Yunfei Lu, Dandan Tu, Bing Qin
| Challenge: | Existing approaches to enhance multilingual reasoning capabilities rely on costly multilingual training or employ prompting with external translation tools. |
| Approach: | They propose a training-free inference-time method to enhance multilingual reasoning capabilities via Representation Engineering without additional training data or tools. |
| Outcome: | The proposed method outperforms existing methods on four reasoning benchmarks in English and Thai and Swahili. |
IndiGEC: Multilingual Grammar Error Correction for Low-Resource Indian Languages (2025.emnlp-main)
Copied to clipboard
| Challenge: | despite growing interest in GEC, most research has focused on English due to the lack of benchmark datasets for low-resource lan-guages. |
| Approach: | They propose a new approach to generate high-quality synthetic data for GEC using monolingual corpora. |
| Outcome: | The proposed framework outperforms other monolingual methods in English, Hindi, Bengali, Marathi, and Tamil. |
Job Unfair: An Investigation of Gender and Occupational Bias in Free-Form Text Completions by LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | a recent study has identified that LLMs are used in domains where they support or replace human decision-making . a systematic review of LLM outputs shows that many facets of social bias remain unaccounted for . |
| Approach: | They propose to disentangle gender and occupational biases in Italian and English as expressed by LLMs. |
| Outcome: | The proposed method captures gender and occupational biases in Italian and English . it also shows that models struggle with gender-neutral expressions, especially beyond English - the authors conclude . |
GottBERT: a pure German Language Model (2024.emnlp-main)
Copied to clipboard
Raphael Scheible, Johann Frei, Fabian Thomczyk, Henry He, Patric Tippmann, Jochen Knaus, Victor Jaravine, Frank Kramer, Martin Boeker
| Challenge: | Pre-trained language models have advanced natural language processing (NLP) despite the introduction of BERT, single-language models are still relevant. |
| Approach: | They present a German singlelanguage RoBERT model pre-trained exclusively on the German portion of the OSCAR dataset. |
| Outcome: | The GottBERT model outperforms the existing models on Named Entity Recognition and text classification tasks. |
FastMCTS: A Simple Sampling Strategy for Data Synthesis (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for generating multi-step reasoning data rely on rejection sampling, which generates trajectories independently and suffers from inefficiency and imbalanced sampling across problems of varying difficulty levels. |
| Approach: | They propose a data synthesis strategy inspired by Monte Carlo Tree Search . it offers step-level evaluation signals and promotes balanced sampling . |
| Outcome: | Experiments show that FastMCTS generates 30% more correct reasoning paths than rejection sampling. |
RoBERTa Low Resource Fine Tuning for Sentiment Analysis in Albanian (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in the education domain have provided new opportunities for solving interesting, but difficult problems. |
| Approach: | They propose to use EduSenti to fine-tune language models for assigning sentiment to reviews of educators' performance annotated for sentiment, emotion and educational topic. |
| Outcome: | The proposed model is compared with an Albanian masked language trained model from the last XLM-RoBERTa checkpoint and shows that it is a good fit for the proposed model. |
Language Fusion for Parameter-Efficient Cross-lingual Transfer (2025.acl-long)
Copied to clipboard
| Challenge: | Limited availability of multilingual text corpora for pretraining results in poor performance on downstream tasks due to undertrained representation spaces for languages other than English. |
| Approach: | They propose a method that integrates source and target language representations within low-rank (LoRA) adapters using lightweight linear transformations to enhance representation quality and transfer performance for languages other than English. |
| Outcome: | The proposed method improves representation quality and performance for languages other than English while maintaining parameter efficiency. |
SlovakSum: A Large Scale Slovak Summarization Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing datasets with hundreds and thousands of documents are mainly in the English language, but the available data is small or non-existent. |
| Approach: | They propose to use a large Slovak news summarization dataset to evaluate its performance . the dataset contains headlines, short abstracts, and full source text . |
| Outcome: | The proposed dataset is compared with a standard ROUGE metric and a mT5 model to evaluate its performance. |
SPLICE: A Singleton-Enhanced PipeLIne for Coreference REsolution (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing attempts to integrate singleton mention detection into end-to-end coreference resolution for English have been hampered by the lack of singletont mention spans in the OntoNotes benchmark. |
| Approach: | They propose a two-step neural mention and coreference resolution system that integrates singleton mentions with OntoNotes syntax trees to achieve a near approximation of the Ontonotes dataset with all singletont mentions. |
| Outcome: | The proposed system achieves 94% recall on a sample of gold singletons. |
Strengthening the WiC: New Polysemy Dataset in Hindi and Lack of Cross Lingual Transfer (2024.lrec-main)
Copied to clipboard
| Challenge: | a new study addresses the problem of natural language processing in low-resource languages such as Hindi . the paper focuses on Word Sense Disambiguation, a fundamental NLP task that deals with polysemous words. |
| Approach: | They propose a Hindi WSD dataset that allows training and testing of contextualized models. |
| Outcome: | The proposed dataset enables training and testing of contextualized models in Hindi . the results show that the proposed dataset can handle polysemy tasks in low-resource languages . |
Modal Dependency Parsing as Structured Prediction over Source-Cue Scope (2026.acl-long)
Copied to clipboard
| Challenge: | Existing work on identifying sources only focuses on defining source-introducing cues . a structured model focuses learning at the source-cue level and constrains event-level decisions to a small, scope-defined candidate set. |
| Approach: | They propose a framework that leverages large language models to explicitly identify source-cue pairs and their respective scope to define modal contexts. |
| Outcome: | The proposed framework surpasses state-of-the-art results by 3 and 4% for English and Chinese datasets. |
How Programming Concepts and Neurons Are Shared in Code Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Several studies have focused on programming languages in a monolingual setting, but most focus on programming language models. |
| Approach: | They perform a few-shot translation task on 21 PL pairs using two Llama-based models and decode the embeddings of intermediate layers. |
| Outcome: | The proposed model assigns high probability to English tokens in the second half of the intermediate layers and language-specific neurons are concentrated in the bottom layers . the model's concept space is closer to English (including PL keywords) and the model is more efficient at identifying language-related neurons. |
BrailleLLM: Braille Instruction Tuning with Large Language Models for Braille Domain Tasks (2025.emnlp-main)
Copied to clipboard
Tianyuan Huang, Zepeng Zhu, Hangdi Xing, Zirui Shao, Zhi Yu, Chaoxiong Yang, Jiaxian He, Xiaozhong Liu, Jiajun Bu
| Challenge: | Existing Braille research focuses on isolated tasks while mixed-content Braille tasks face data scarcity and ambiguities. |
| Approach: | They propose a syntax tree-based augmentation method tailored for Braille data. |
| Outcome: | The proposed method improves Braille translation, formula-to-Braille conversion, and mixed-text translation. |
Triples-to-isiXhosa (T2X): Addressing the Challenges of Low-Resource Agglutinative Data-to-Text Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing data-to-text models are designed for the linguistic typology of English, but they are not suitable for low-resource languages. |
| Approach: | They propose a new dataset based on a subset of WebNLG that is agglutinative and low-resource data-to-text. |
| Outcome: | The proposed model outperforms existing models for isiXhosa and Finnish and fine-tunes machine translation models as the best method overall. |
Truth Knows No Language: Evaluating Truthfulness Beyond English (2025.acl-long)
Copied to clipboard
Blanca Calvo Figueras, Eneko Sagarzazu, Julen Etxaniz, Jeremy Barnes, Pablo Gamallo, Iria de-Dios-Flores, Rodrigo Agerri
| Challenge: | a new benchmark evaluates the truthfulness of large language models (LLMs) based on imitative falsehoods. |
| Approach: | They propose a professionally translated extension of the TruthfulQA benchmark . it evaluates truthfulness in Basque, Catalan, Galician, and Spanish . |
| Outcome: | The proposed extension of the TruthfulQA benchmark evaluates truthfulness in Basque, Catalan, Galician, and Spanish. |
Multilingual Refusal Alignment for Safer Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used globally, but their safety and alignment can vary unpredictably between languages. |
| Approach: | They propose a multilingual refusal alignment dataset to investigate whether alignment transfers cross-lingually and how language consistency is preserved during training. |
| Outcome: | The proposed model can be trained on multilingual datasets without affecting general performance. |
Zero-shot Learning for Multilingual Discourse Relation Classification (2024.lrec-main)
Copied to clipboard
| Challenge: | Discourse analysis is a hard task, but data is limited for other languages. |
| Approach: | They propose to use zero-shot learning to combine discourse relation data . they compare two versions of the same text with different labels . |
| Outcome: | The proposed method can be applied to languages, frameworks, or similarity measures. |
Explicit Learning and the LLM in Machine Translation (2025.emnlp-main)
Copied to clipboard
| Challenge: | a growing number of researchers are examining whether large language models can learn to translate a "new" language using grammar books. |
| Approach: | They examine an LLM's ability to learn new languages using grammar books . authors suggest alternative fine-tuning strategies to improve explicit learning . |
| Outcome: | The proposed model can learn low-resource languages described in grammar books but lacking extensive corpora. |
AfroCS-xs: Creating a Compact, High-Quality, Human-Validated Code-Switched Dataset for African Languages (2025.acl-long)
Copied to clipboard
Kayode Olaleye, Arturo Oncevay, Mathieu Sibue, Nombuyiselo Zondi, Michelle Terblanche, Sibongile Mapikitla, Richard Lastrucci, Charese Smiley, Vukosi Marivate
| Challenge: | AfroCS-xs is a low-quality dataset for code-switching in multilingual communities . code-witching is prevalent in multicultural societies but lacks high-quality data for model development . |
| Approach: | They propose to use human-validated synthetic code-switched datasets to generate code-witched sentences for four African languages and English within a specific domain—agriculture. |
| Outcome: | The proposed model improves translation accuracy on the high-quality dataset for four African languages and English within a specific domain—agriculture. |
KRETA: A Benchmark for Korean Reading and Reasoning in Text-Rich VQA Attuned to Diverse Visual Contexts (2025.emnlp-main)
Copied to clipboard
| Challenge: | Understanding and reasoning over text within visual contexts poses a significant challenge for Vision-Language Models. |
| Approach: | They propose a benchmark for Korean Reading and rEasoning in Text-rich VQA Attuned to diverse visual contexts to address this challenge. |
| Outcome: | The proposed benchmark is tailored for Korean reading and rEasoning in text-rich VQA attuned to diverse visual contexts. |
UPDESH: Synthesizing Grounded Instruction Tuning Data for 13 Indic Languages (2026.acl-long)
Copied to clipboard
Pranjal A Chitale, Varun Gumma, Sanchit Ahuja, Prashant Kodali, Manan Uppadhyay, Deepthi Sudharsan, Sunayana Sitaram
| Challenge: | Developing culturally grounded multilingual AI systems is challenging for low-resource languages . synthetic data is underexplored, but its effectiveness in multilingual and multicultural contexts is understudied . |
| Approach: | They propose a top-up synthetic data generation framework grounded in Wikipedia content . they use 9.5M data points across 13 Indian languages and English to generate a high-quality dataset . |
| Outcome: | The proposed model improves on NLG tasks and narrows performance gaps with high-resource languages. |