Papers by Matthew Gombolay
FedPerC: Federated Learning for Language Generation with Personal and Context Preference Embeddings (2023.findings-eacl)
Copied to clipboard
| Challenge: | federated learning is a decentralized learning paradigm that assumes no access to a large labeled dataset and instead leverages averaged parameter updates across all users of the system. |
| Approach: | They propose a method to personalize federated learning with personal embeddings and shared context embeddables. |
| Outcome: | The proposed approach achieves 50% improvement in test-time perplexity using 0.001% of the memory required by baseline approaches and greater sample- and compute-efficiency. |
Towards a Comprehensive Understanding and Accurate Evaluation of Societal Biases in Pre-Trained Transformers (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing pre-trained language models are not fully considered for societal biases . pre-training models can be useful for many NLP tasks, but they can be harmful when used at scale. |
| Approach: | They investigate gender and racial bias across pre-trained language models . they evaluate bias within pre-trainers using three metrics: WEAT, sequence likelihood, and pronoun ranking. |
| Outcome: | The proposed model fails to detect gender and racial biases in pre-trained models . the model is ineffective when word embedding, demonstrating the need for more robust bias testing in transformers. |
A Computational Interface to Translate Strategic Intent from Unstructured Language in a Low-Data Setting (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on interpreting the high-level strategic intent of humans has focused on enabling them to specify their strategy to an AI system. |
| Approach: | They propose to translate unstructured language strategies into actionable intent in the form of goals and constraints by using a game environment to build a computational interface. |
| Outcome: | The proposed model significantly outperforms human interpreters in inferring strategic intent from language (p 0.05) and in a low-data setting. |