Papers with FSAM
Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning pretrained language models suffer from poor generalization . however, they add a perturbation to each model parameter equally, which is sub-optimal . |
| Approach: | They propose a sharpness-aware minimization optimization procedure that introduces a Fisher mask to improve the efficiency of SAM. |
| Outcome: | The proposed method outperforms the vanilla sharpness-aware minimization method on GLUE and SuperGLUE benchmarks. |
Finite State Machine Pattern-Root Arabic Morphological Generator, Analyzer and Diacritizer (2020.lrec-1)
Copied to clipboard
| Challenge: | Using a finite-state morphologizer, we generate and analyze undiacritized Modern Standard Arabic (MSA) words. |
| Approach: | They propose to use a finite-state Arabic Morphologizer to generate and analyze undiacritized Arabic words and diacritize them. |
| Outcome: | The proposed model generates and analyzes undiacritized Modern Standard Arabic (MSA) words and diacritizes them. |