| Challenge: | Transfer learning is a form of learning that adapts a model trained on data-rich sources to low-resource targets. |
| Approach: | They propose a source valuation framework that quantifies the usefulness of the sources in transfer learning by using the Shapley value method. |
| Outcome: | The proposed framework is effective in choosing useful transfer sources and the source values match the intuitive source-target similarity. |
Similar Papers
To Share or not to Share: Predicting Sets of Sources for Model Transfer Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to select transfer sources are limited by text and task similarity, which limits their application in transfer settings where both the task and the text domain change. |
| Approach: | They propose a model similarity measure that represents text and task similarity jointly to automatically determine which and how many sources to exploit. |
| Outcome: | The proposed approach improves performance by 24 F1 points for predicting promising sources across domains and tasks with similar models. |
Data Selection for Fine-tuning Large Language Models Using Transferred Shapley Values (2023.acl-srw)
Copied to clipboard
| Challenge: | Large language models (LMs) have been shown to be highly effective for identifying harmful training instances, but dataset size and model complexity constraints limit the ability to apply Shapley-based data valuation to fine-tuning large pre-trained language models. |
| Approach: | They propose an algorithm that aggregates Shapley values from subsets for valuation of entire training set and a value transfer method that leverages value information extracted from a simple classifier trained using representations from the target language model. |
| Outcome: | The proposed method outperforms existing methods on benchmark datasets and can filter fine-tuning data to increase language model performance compared to training with the full fine-uning dataset. |
Multi-Source Cross-Lingual Model Transfer: Learning What to Share (P19-1)
Copied to clipboard
| Challenge: | Cross-lingual transfer learning (CLTL) is a viable method for building NLP models for a low-resource target language . however, many languages lack the labeled training data necessary for training deep neural nets for varying NLP tasks. |
| Approach: | They propose a cross-lingual transfer learning method that leverages annotated data from other languages to build NLP models for a target language. |
| Outcome: | The proposed model achieves significant performance gains over prior art over multiple text classification and sequence tagging tasks including a large-scale industry dataset. |
Exploring and Predicting Transferability across NLP Tasks (2020.emnlp-main)
Copied to clipboard
Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, Mohit Iyyer
| Challenge: | Recent advances in NLP demonstrate the effectiveness of training large-scale language models and transferring them to downstream tasks. |
| Approach: | They conduct an extensive study of the transferability between 33 NLP tasks across three broad classes of problems. |
| Outcome: | The proposed model can improve performance even with low-data source tasks that differ substantially from the target task. |
Quality Scoring of Source Words in Neural Translation Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent approaches to improving word-level quality scores on input source sentences require training special word-scoring models or require repeated invocation of the translation model. |
| Approach: | They propose to reason how well each word is explained by the target sentence as against the source language model and use it to translate into an unfamiliar target language. |
| Outcome: | The proposed method provides up to five points higher F1 scores and is significantly faster than the state of the art methods on three language pairs. |
Choosing Transfer Languages for Cross-Lingual Learning (P19-1)
Copied to clipboard
Yu-Hsiang Lin, Chian-Yu Chen, Jean Lee, Zirui Li, Yuyan Zhang, Mengzhou Xia, Shruti Rijhwani, Junxian He, Zhisong Zhang, Xuezhe Ma, Antonios Anastasopoulos, Patrick Littell, Graham Neubig
| Challenge: | Cross-lingual transfer is a useful tool for improving performance of natural language processing (NLP) on low-resource languages. |
| Approach: | They propose to use cross-lingual transfer to improve accuracy of low-resource languages . they build models that consider features to perform prediction on such languages based on ranking problem . |
| Outcome: | The proposed model predicts good transfer languages much better than baselines considering single features in isolation. |
Explaining Pre-Trained Language Models with Attribution Scores: An Analysis in Low-Resource Settings (2024.lrec-main)
Copied to clipboard
| Challenge: | Currently, prompt-based models are gaining popularity due to their easier adaptability in low-resource settings. |
| Approach: | They analyze attribution scores extracted from prompt-based models w.r.t. plausibility and faithfulness and compare them with attribution score extracted from fine-tuned models and large language models. |
| Outcome: | The proposed model outperforms attention and Integrated Gradients in plausibility and faithfulness, while fine-tuning models are harder to explain in low-resource settings. |
Lost in the Source Language: How Large Language Models Evaluate the Quality of Machine Translation (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that Large Language Models (LLMs) can be used as translation evaluators. |
| Approach: | They propose to use both coarse-grained and fine-grounded prompts to discern the utility of source versus reference data in machine translation evaluation tasks. |
| Outcome: | The proposed model can be used to evaluate translations in multiple languages. |
From Scoring to Explanations: Evaluating SHAP and LLM Rationales for Rubric-based Teaching Quality Assessment (2026.findings-acl)
Copied to clipboard
Ivo Bueno, Babette Bühler, Philipp Stark, Tim Fütterer, Ulrich Trautwein, Dorottya Demszky, Heather Hill, Enkelejda Kasneci
| Challenge: | a framework for sentence-level interpretability of rubric-based scoring is proposed . aaron e. smith: automated scoring models provide little insight into why scores are produced . |
| Approach: | They propose a framework for sentence-level interpretability of rubric-based scoring that combines Shapley-value attributions with rationales generated by large language models. |
| Outcome: | The proposed framework compares fine-tuned pretrained language models with large language models . it shows that fine- tuned models outperform LLMs in prediction accuracy but exhibit label compression toward mid-scale scores . |
Less is More: Parameter-Efficient Selection of Intermediate Tasks for Transfer Learning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Prior methods producing useful task rankings are infeasible for large source pools . Embedding space maps (ESMs) reduce execution time and disk space usage . |
| Approach: | They introduce Embedded Space Maps (ESMs) that approximate the effect of fine-tuning a language model. |
| Outcome: | The proposed method reduces execution time and disk space usage by 10 and 278, respectively, while retaining high selection performance. |