Papers with Scalarization
Upsample or Upweight? Balanced Training on Heavily Imbalanced Datasets (2025.naacl-long)
Copied to clipboard
| Challenge: | a lack of data across domains creates significant imbalances in training data sizes . a recent study shows that temperature sampling and scaling are equivalent but differ under stochastic gradient descent due to differences in gradient variance. |
| Approach: | They propose a method that upsamples low-resource languages and upweights their loss functions to address this disparity. |
| Outcome: | The proposed method competes effectively with existing data re-weighting techniques while offering computational efficiency. |