Papers by Demi Guo
Sequence-Level Mixed Sample Data Augmentation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Despite their empirical success, neural networks still have difficulty capturing compositional aspects of natural language. |
| Approach: | They propose a data augmentation approach to encourage compositional behavior in neural networks . they propose to softly combine input/output sequences from the training set . |
| Outcome: | The proposed approach yields 1.0 BLEU improvement on translation datasets over baselines. |
Parameter-Efficient Transfer Learning with Diff Pruning (2021.acl-long)
Copied to clipboard
| Challenge: | Pretrained networks are difficult to deploy for multiple tasks in storage-constrained settings. |
| Approach: | Diff pruning enables parameter-efficient transfer learning that scales well with new tasks. |
| Outcome: | Diff pruning can match the performance of finetuned baselines on the GLUE benchmark while only modifying 0.5% of the pretrained model’s parameters per task. |