Papers by Hugh McMahan
Can Public Large Language Models Help Private Cross-device Federated Learning? (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have shown that public data can be used to improve privacy-utility trade-offs for large and small language models. |
| Approach: | They propose to use large-scale public data to help differentially private FL training . they propose a distribution matching algorithm with theoretical grounding to sample public data close to private data distribution . |
| Outcome: | The proposed method is efficient and effective for training private models by taking advantage of public data. |
A Hassle-free Algorithm for Strong Differential Privacy in Federated Learning Systems (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Differential privacy (DP) and federated learning (FL) are used for language models training in production mobile keyboard applications. |
| Approach: | They propose a variant of DP-FTRL that uses a correlated noise mechanism to train on-device language models. |
| Outcome: | The proposed method improves privacy-utility trade-off and memory efficiency over existing FL methods while simplifying usage requirements and reducing memory. |