Memorization vs. Generalization : Quantifying Data Leakage in NLP Performance Evaluation (2021.eacl-main)
Copied to clipboard
| Challenge: | Public datasets are often used to evaluate the efficacy and generalizability of state-of-the-art methods for many tasks in natural language processing (NLP). |
| Approach: | They identify leakage of training data into test data on several publicly available datasets used to evaluate NLP tasks, including named entity recognition and relation extraction. |
| Outcome: | The proposed model can memorize and generalize data on several publicly available datasets and is compared against previously unseen data. |
Similar Papers
Revisiting the Effects of Leakage on Dependency Parsing (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent work shows that treebank size and linguistic variation are important factors that explain the variation in dependency parsing performance. |
| Approach: | They propose a measure of leakage that explains and correlates with observed performance variation. |
| Outcome: | The proposed measure explains and correlates with observed performance variation. |
Principles from Clinical Research for NLP Model Generalization (2024.naacl-long)
Copied to clipboard
| Challenge: | In clinical research, generalizability depends on (a) internal validity of experiments and (b) external validity or transportability of the results to the wider population. |
| Approach: | They propose to ensure internal validity when building machine learning models in NLP by incorporating learning spurious correlations into their models. |
| Outcome: | The proposed model can perform well on data unseen during training, but drawn from the same distribution or population. |
Memorisation versus Generalisation in Pre-trained Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | State-of-the-art pre-trained language models have been shown to memorise facts and perform well with limited amounts of training data. |
| Approach: | They propose to extend pre-trained language models to generalise and memorise facts in noisy and low-resource scenarios. |
| Outcome: | The proposed extension improves performance in low-resource named entity recognition tasks. |
Separating Retention from Extraction in the Evaluation of End-to-end Relation Extraction (2021.emnlp-main)
Copied to clipboard
| Challenge: | State-of-the-art NLP models adopt shallow heuristics that limit their generalization capability. |
| Approach: | They propose to use heuristics that limit their generalization capability to model lexical overlap with the training set in Named-Entity Recognition and Event or Type heuristic in Relation Extraction to test their models. |
| Outcome: | The proposed model can perform better on the two key tasks, while the retention of training relation triples. |
Investigating How Pre-training Data Leakage Affects Models’ Reproduction and Detection Capabilities (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies do not examine how leaked instances in training datasets influence LLMs’ output and detection capabilities. |
| Approach: | They conduct an experimental survey to examine the relationship between data leakage in training datasets and its effects on the generation and detection by Large Language Models (LLMs). |
| Outcome: | The results show that enhancing leakage detection through few-shot learning can help mitigate the impact of the leakage rate in the training data on detection performance. |
Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs (2024.eacl-long)
Copied to clipboard
| Challenge: | Lack of access to model details has raised concerns about data contamination among researchers. |
| Approach: | They conduct the first systematic analysis of work using OpenAI’s GPT-3.5 and GPT-4, the most prominently used LLMs today, in the context of data contamination. |
| Outcome: | The proposed models have been exposed to 4.7M samples from 263 benchmarks during the first year after their release. |
Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Considering the vast size and wide-ranging sources of LLMs’ training data, it could explicitly or implicitly include test data. |
| Approach: | They propose a Contamination Detection via output Distribution (CDD) which detects data contamination only by identifying the peakedness of LLM's output distribution. |
| Outcome: | The proposed method improves performance by 21.8%-30.2% on humanEval and TED: trustworthy evaluation via output distribution. |
Data Contamination: From Memorization to Exploitation (2022.acl-short)
Copied to clipboard
| Challenge: | Pretrained language models are typically trained on web-based datasets that are often "contaminated" with downstream test sets. |
| Approach: | They propose a method to pretrain BERT models on Wikipedia and labeled downstream datasets and fine-tune them on the relevant task. |
| Outcome: | The proposed method compares models on Wikipedia and labeled downstream datasets on two models and three downstream tasks. |
Does it Really Generalize Well on Unseen Data? Systematic Evaluation of Relational Triple Extraction Methods (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing extraction models memorize and recall already seen triples but cannot generalize effectively for unseen triples. |
| Approach: | They propose a method to generalize existing extraction models by rearranging datasets and augmenting test sets. |
| Outcome: | The proposed method can significantly increase the generalization performance of existing models. |
Out-of-Distribution Generalization in Natural Language Processing: Past, Present, and Future (2023.emnlp-main)
Copied to clipboard
Linyi Yang, Yaoxian Song, Xuan Ren, Chenyang Lyu, Yidong Wang, Jingming Zhuo, Lingqiao Liu, Jindong Wang, Jennifer Foster, Yue Zhang
| Challenge: | Existing literature on the generalization of machine learning models to out-of-distribution data is lacking. |
| Approach: | They propose to present the first comprehensive review of recent progress, methods, and evaluations on the generalization challenge from an OOD perspective in natural language understanding. |
| Outcome: | The proposed survey provides the first comprehensive review of recent progress, methods, and evaluations on the generalization challenge from an OOD perspective in natural language understanding. |