| Challenge: | Having a parallel dataset for Natural Language Inference in Dutch is problematic for some NLP systems. |
| Approach: | They propose to translate a SICK dataset from English into Dutch to compare models for both languages. |
| Outcome: | The proposed dataset compares models on English and Dutch on two tasks. |
Similar Papers
Assessing Monotonicity Reasoning in Dutch through Natural Language Inference (2023.findings-eacl)
Copied to clipboard
| Challenge: | a novel dataset for natural language inference (NLI) is used to study monotonicity reasoning in Dutch. |
| Approach: | They investigate monotonicity reasoning in Dutch using a novel dataset . they find that models struggle with downward entailing contexts . |
| Outcome: | The proposed dataset shows that models struggle with downward entailing contexts, and argue that this is due to a poor understanding of negation. |
SI-NLI: A Slovene Natural Language Inference Dataset and Its Evaluation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing datasets for natural language inference (NLI) are limited to English and a few other well-resourced languages. |
| Approach: | They propose to use a dataset for natural language inference to extend the resources for the task. |
| Outcome: | The proposed dataset is constructed from scratch using knowledgeable annotators with carefully crafted guidelines aiming to avoid common problems in existing datasets. |
MTEB-NL and E5-NL: Embedding Benchmark and Models for Dutch (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in embedding resources have led to a lack of representation of the Dutch language in multilingual resources. |
| Approach: | They introduce Massive Text Embedding Benchmark for Dutch (MTEB-NL) which includes existing Dutch datasets and newly created ones, covering a wide range of tasks. |
| Outcome: | The proposed models demonstrate strong performance across multiple tasks. |
DUMB: A Benchmark for Smart Evaluation of Dutch Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Current Dutch monolingual models under perform and suggest training larger models with other architectures and pre-training objectives. |
| Approach: | They propose a Dutch Model Benchmark that compares performance of language models to a strong baseline that can be referred to in the future even when assessing different sets of language model. |
| Outcome: | The proposed benchmark compares the performance of 14 pre-trained language models to a strong baseline . the results suggest training larger models with other architectures and pre-training objectives . |
NLI4CT: Multi-Evidence Natural Language Inference for Clinical Trial Reports (2023.emnlp-main)
Copied to clipboard
| Challenge: | Clinical trial reports (CTRs) are indispensable for the development of personalized medicine. |
| Approach: | They propose a resource to help researchers interpret clinical trial reports . they use natural language inference to compute textual entailment . |
| Outcome: | The proposed resource is the first to cover interpretation of full clinical trial reports . it includes tasks to determine inference relation between natural language statements and CTRs . |
InferBR: A Natural Language Inference Dataset in Portuguese (2024.lrec-main)
Copied to clipboard
| Challenge: | Portuguese has few NLI-annotated datasets created through automatic translation followed by manual checking. |
| Approach: | They propose to generate premises and hypotheses using a semiautomatic process to generate sentences and manually check the annotations. |
| Outcome: | The proposed dataset is better at recognizing entailment classes in other Portuguese datasets than the reverse. |
Baselines and Test Data for Cross-Lingual Inference (L18-1)
Copied to clipboard
| Challenge: | Recent research on textual entailment is limited to English, but it is expanding to other languages. |
| Approach: | They propose to extend the research in SNLI-style natural language inference toward multilingual evaluation by using cross-lingual word embeddings and machine translation. |
| Outcome: | The proposed system scores an average accuracy of just over 75%, but it is not perfect. |
Data and Representation for Turkish Natural Language Inference (2020.emnlp-main)
Copied to clipboard
| Challenge: | Large annotated datasets in NLP are overwhelmingly in English . obtaining new annotation resources for each task in each language would be prohibitively expensive . |
| Approach: | They propose to use machine translation to translate large annotated datasets into Turkish . they find that in-language embeddings are essential and morphological parsing can be avoided . |
| Outcome: | The proposed model trains on human-translated evaluation sets. |
A synthetic data approach for domain generalization of NLI models (2024.acl-long)
Copied to clipboard
| Challenge: | Natural Language Inference (NLI) datasets are important benchmark tasks for LLMs . however, their realistic performance on out-of-distribution/domain data is less well-understood . a T5-small model trained with our data improves around 7% on average compared to the best alternative dataset . |
| Approach: | They propose a new approach for generating NLI data in diverse domains and lengths . they show that models trained on this data have the best generalization to completely new downstream test settings . |
| Outcome: | The proposed model can be trained on datasets with high-quality examples with meaningful premises and high accuracy. |
Beyond Monolingual Assumptions: A Survey on Code-Switched NLP in the Era of Large Language Models across Modalities (2026.acl-long)
Copied to clipboard
| Challenge: | Amidst the rapid advances of large language models, most LLMs struggle with mixed-language inputs, limited Code-switching datasets, and evaluation biases. |
| Approach: | They propose a roadmap for inclusive datasets, fair evaluation, and linguistically grounded models to achieve truly multilingual intelligence. |
| Outcome: | The proposed frameworks are based on 327 studies spanning five research areas, 15+ NLP tasks, 30+ datasets, and 80+ languages. |