On Sensitivity of Learning with Limited Labelled Data to the Effects of Randomness: Impact of Interactions and Systematic Choices (2024.emnlp-main)
Copied to clipboard
| Challenge: | Using limited labelled data, learning with limited labels is sensitive to the effects of uncontrolled randomness. |
| Approach: | They propose to investigate the effects of individual randomness factors while taking the interactions between them into consideration. |
| Outcome: | The proposed method mitigates the effects of other factors while observing how the performance varies across multiple runs. |
Similar Papers
Understanding the effects of language-specific class imbalance in multilingual fine-tuning (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods to fine-tune large language models have been developed to reduce the amount of resources needed to perform classification tasks. |
| Approach: | They modify traditional class weighing approach to reduce imbalance by calculating class weights separately for each language. |
| Outcome: | The proposed model improves performance and reduces the promotion of uninformative features. |
Stubborn Lexical Bias in Data and Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has focused on spurious correlations between features and labels in training data . but, we find strong evidence of corresponding bias in the trained models . |
| Approach: | They propose a method to reduce spurious correlations in training data by reweighting it using a large pool of extracted features. |
| Outcome: | The proposed method reduces spurious correlations in training data, but still finds strong evidence of bias in trained models. |
Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classification (2021.emnlp-main)
Copied to clipboard
| Challenge: | Fine-grained classification tasks involve distinguishing between classes with subtle differences between them. |
| Approach: | They analyse fine-grained text classification tasks by embedding class relationships into a contrastive objective function to help differently weigh the positives and negatives. |
| Outcome: | The proposed model outperforms previous contrastive methods on emotion classification and sentiment analysis. |
On the Importance of Data Size in Probing Fine-tuned Models (2022.findings-acl)
Copied to clipboard
| Challenge: | Several studies have investigated the reasons behind the effectiveness of fine-tuning, usually through the lens of probing. |
| Approach: | They propose to investigate the reasons behind the effectiveness of fine-tuning by examining the impact of data size on the extent of encoded linguistic knowledge. |
| Outcome: | The proposed probes show that the size of the training data affects the recoverability of the changes made to the model’s linguistic knowledge. |
Learning Concept Abstractness Using Weak Supervision (D18-1)
Copied to clipboard
Ella Rabinovich, Benjamin Sznajder, Artem Spector, Ilya Shnayderman, Ranit Aharonov, David Konopnicki, Noam Slonim
| Challenge: | Existing methods for inferring abstractness of words and expressions without labeled data are limited and limited. |
| Approach: | They propose a weakly supervised approach for inferring the property of abstractness of words and expressions in the absence of labeled data. |
| Outcome: | The proposed approach obtains high correlation with human labels in the absence of labeled data. |
Cognitive Effects and Biases in Large Language Models (2026.eacl-tutorials)
Copied to clipboard
| Challenge: | This tutorial bridges psychology and NLP to clarify cognitive effects and biases in large language models. |
| Approach: | This tutorial bridges psychology and NLP to clarify cognitive effects and biases in large language models. |
| Outcome: | This tutorial bridges psychology and NLP to clarify cognitive effects and biases in large language models. |
Label Representations in Modeling Classification as Text Generation (2020.aacl-srw)
Copied to clipboard
| Challenge: | Existing methods for text generation use strings to represent labels . linguistic properties of labels do affect performance, though their results are limited to document retrieval. |
| Approach: | They investigate the effect of string representations on how effectively a model learns a task . they use four standard text classification tasks to model string representation . |
| Outcome: | The proposed model improves on four standard text classification tasks . the results are largely negative in the low data setting . |
Beyond Demographics: Fine-tuning Large Language Models to Predict Individuals’ Subjective Text Perceptions (2025.acl-long)
Copied to clipboard
| Challenge: | Recent work has shown that LLMs perform poorly when prompted with sociodemographic attributes, suggesting limited inherent sociodemography knowledge. |
| Approach: | They propose to train large language models to be accurate sociodemographic models of annotator variation by using a curated dataset of five tasks with standardized sociodemography. |
| Outcome: | The proposed models improve in sociodemographic prompting when trained but this performance gain is largely due to models learning annotator-specific behaviour rather than sociodemography. |
What Really Matters for Table LLMs? A Meta-Evaluation of Model and Data Effects (2026.findings-eacl)
Copied to clipboard
Naihao Deng, Sheng Zhang, Henghui Zhu, Shuaichen Chang, Jiani Zhang, Alexander Hanbo Li, Chung-Wei Hang, Hideo Kobayashi, Yiqun Hu, Patrick Ng
| Challenge: | a series of paradigm shifts have come with distinct characteristics and challenges associated with table modeling. |
| Approach: | They propose to replicate four table LLMs by instruction-tuning three foundation models on four existing datasets. |
| Outcome: | The results show that base model choice plays a more dominant role than training data itself. |
Unveiling the Generalization Power of Fine-Tuned Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated exceptional multitasking abilities, but the comprehensive effects of fine-tuning on the LLMs’ generalization ability are not fully understood. |
| Approach: | They conduct extensive experiments across five distinct language tasks on different datasets to investigate whether fine-tuning affects the generalization ability intrinsic to LLMs. |
| Outcome: | The proposed model can generalize to different domains and tasks by integrating the in-context learning strategy during fine-tuning on generation tasks. |