Papers with pre-finetuning
Enhancing Society-Undermining Disinformation Detection through Fine-Grained Sentiment Analysis Pre-Finetuning (2024.findings-eacl)
Copied to clipboard
| Challenge: | a new method for disinformation detection is needed to address the issue of disinformation, authors argue . a series of rigorous experiments establishes a notable connection between disinformation and fine-grained sentiment labels . |
| Approach: | They propose a method leveraging pre-finetuning concept for efficient detection and removal of disinformation that may undermine society. |
| Outcome: | The proposed method improves performance across languages and languages, showing promising results. |
Multi-Task Pre-Finetuning of Lightweight Transformer Encoders for Text Classification and NER (2025.emnlp-industry)
Copied to clipboard
| Challenge: | nave multitask pre-finetuning introduces conflicting optimization signals that degrade overall performance. |
| Approach: | They propose a framework that enables a single shared encoder backbone with modular adapters. |
| Outcome: | The proposed framework achieves comparable performance to individual pre-finetuning while meeting practical deployment constraint. |
Muppet: Massive Multi-task Representations with Pre-Finetuning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows gains from pre-training and fine-tuning that are multi-task . but it can be difficult to know which intermediate tasks will best transfer . |
| Approach: | They propose a large-scale learning stage for pre-finetuning between pre-training and fine-tun. |
| Outcome: | The proposed model improves performance on pretrained discriminators and generation models on a wide range of tasks while improving sample efficiency during fine-tuning. |
Investigating the Effect of Pre-finetuning BERT Models on NLI Involving Presuppositions (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a study of presupposition, discourse and sarcasm suggests that pre-finetuning can improve models' performance on presimplified cases. |
| Approach: | They propose to leverage the connection between presupposition, discourse and sarcasm to improve models' performance. |
| Outcome: | The proposed model improves on cases involving presupposition by pre-finetuning on additional tasks and datasets. |
REZE: Representation Regularization for Domain-adaptive Text Embedding Pre-finetuning (2026.acl-long)
Copied to clipboard
| Challenge: | Recent text embedding models often introduce task-induced bias alongside domain knowledge, leading to performance degradation. |
| Approach: | They propose a representation regularization framework that explicitly controls representation shift during embedding pre-finetuning. |
| Outcome: | The proposed framework outperforms standard pre-finetuning and isotropy-oriented post-hoc regularization in most settings. |