Chenghong Wang, Jieren Deng, Xianrui Meng, Yijue Wang, Ji Li, Sheng Lin, Shuo Han, Fei Miao, Sanguthevar Rajasekaran, Caiwen Ding
| Challenge: | Existing FL frameworks require a trusted aggregator or require heavy-weight cryptographic primitives, which makes the performance significantly degraded. |
| Approach: | They propose a framework that is federated and efficient for NLP . they propose to eliminate the need for trusted entities and achieve better model accuracy . |
| Outcome: | The proposed framework achieves better model accuracy and model accuracy than existing FL frameworks. |
Similar Papers
Safely Learning with Private Data: A Federated Learning Framework for Large Language Model (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing large language models (LLMs) use large amounts of public data and massive parameters, but private data is often stored in isolated data silos. |
| Approach: | They propose a Federated Learning framework for large language models which offloads most training parameters to the server while training embedding and output layers locally. |
| Outcome: | The proposed framework achieves comparable metrics to centralized chatGLM model on NLU and generation tasks. |
FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Tasks (2022.findings-naacl)
Copied to clipboard
Bill Yuchen Lin, Chaoyang He, Zihang Ze, Hulin Wang, Yufen Hua, Christophe Dupuy, Rahul Gupta, Mahdi Soltanolkotabi, Xiang Ren, Salman Avestimehr
| Challenge: | Increasing concerns and regulations about data privacy necessitate the study of privacy-preserving, decentralized learning methods for natural language processing tasks. |
| Approach: | They propose a framework for evaluating federated learning methods on four different tasks . they propose federation between Transformer-based language models and FL methods . |
| Outcome: | The proposed framework compares FL methods on four different tasks under non-IID partitioning strategies. |
Empirical Studies of Institutional Federated Learning For Natural Language Processing (2020.findings-emnlp)
Copied to clipboard
| Challenge: | federated learning is a promising ideology to unite isolated datasets for machine learning problems. |
| Approach: | They propose to use federated natural language processing networks to train a popular NLP model with applications in sentence intent classification. |
| Outcome: | The proposed model is sensitive to imbalanced data load and tested against a federated model under imbalanced datasets. |
Safe-FedLLM: Delving into the Safety of Federated Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing work on federated learning for large language models (FL) addresses privacy and data-silo issues in the training of large language model training. |
| Approach: | They propose a probe-based defense framework for FedLLM that constructs defenses across three levels: Step-Level, Client-Level and Shadow-Level. |
| Outcome: | The proposed framework improves FedLLM's robustness against malicious clients while maintaining competitive performance on benign data. |
Privacy-Preserving Natural Language Processing (2023.eacl-tutorials)
Copied to clipboard
| Challenge: | This tutorial will help the NLP community to get familiar with current research in privacy-preserving methods. |
| Approach: | This tutorial will help the NLP community to get familiar with current research in privacy-preserving methods. |
| Outcome: | The tutorial will cover membership inference, differential privacy, homomorphic encryption, or federated learning, all with typical use-cases and potential pitfalls. |
Efficient-FedRec: Efficient Federated Learning Framework for Privacy-Preserving News Recommendation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing news recommendation methods rely on centralized storage of user click behavior data, which may lead to privacy concerns and hazards. |
| Approach: | They propose a federated learning framework for privacy-preserving news recommendation . they propose aggregation of news representations and user model by a client . |
| Outcome: | The proposed framework reduces computation and communication cost on clients while keeping promising model performance. |
Practical Takes on Federated Learning with Pretrained Language Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | federated learning with pretrained language models for language tasks entails data privacy constraints when learning from diverse data domains. |
| Approach: | They propose to use pretrained language models to learn from diverse data domains . they elaborate hypotheses over the components in federated NLP architectures based on three tasks . |
| Outcome: | The proposed model can generalize by adapting to the different domains. |
Federated Model Decomposition with Private Vocabulary for Text Classification (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train federated learning (FL) for natural language processing require sensitive data to leave local devices. |
| Approach: | They propose a fedrated model decomposition method that protects the privacy of vocabularies . they propose an adaptive updating technique to improve the performance of local models . |
| Outcome: | The proposed method protects the privacy of vocabularies in federated learning tasks . it maintains competitive performance and provides better privacy-preserving capacity compared to status quo methods. |
FedPETuning: When Federated Learning Meets the Parameter-Efficient Tuning Methods of Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing research on federated learning (FL) for pre-trained language models (PLMs) with increasing concerns about data privacy, enterprises or institutions are not allowed to collect data from end devices or local clients to a centralized server for fine-tuning PLMs. |
| Approach: | They investigate the parameter-efficient tuning of pre-trained language models (PLMs) and develop a federated benchmark for four representative PETuning methods . |
| Outcome: | The proposed method can defend against privacy attacks and maintain acceptable performance with reducing heavy resource consumption. |
Privacy-Preserving Federated Learning for Hate Speech Detection (2025.naacl-srw)
Copied to clipboard
| Challenge: | a federated learning system with differential privacy is tailored to low-resource languages . data with fewer than 20 sentences per client struggled due to excessive noise . |
| Approach: | They propose a federated learning system with differential privacy for hate speech detection . they fine-tuned pre-trained language models to find it to be the most effective . |
| Outcome: | The proposed learning system outperforms other models in low-resource languages . balanced datasets and augmenting hateful data with non-hateful examples proved critical . |