Papers by Andrew Silva
FedPerC: Federated Learning for Language Generation with Personal and Context Preference Embeddings (2023.findings-eacl)
Copied to clipboard
| Challenge: | federated learning is a decentralized learning paradigm that assumes no access to a large labeled dataset and instead leverages averaged parameter updates across all users of the system. |
| Approach: | They propose a method to personalize federated learning with personal embeddings and shared context embeddables. |
| Outcome: | The proposed approach achieves 50% improvement in test-time perplexity using 0.001% of the memory required by baseline approaches and greater sample- and compute-efficiency. |
Towards a Comprehensive Understanding and Accurate Evaluation of Societal Biases in Pre-Trained Transformers (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing pre-trained language models are not fully considered for societal biases . pre-training models can be useful for many NLP tasks, but they can be harmful when used at scale. |
| Approach: | They investigate gender and racial bias across pre-trained language models . they evaluate bias within pre-trainers using three metrics: WEAT, sequence likelihood, and pronoun ranking. |
| Outcome: | The proposed model fails to detect gender and racial biases in pre-trained models . the model is ineffective when word embedding, demonstrating the need for more robust bias testing in transformers. |