Federated Continual Learning for Text Classification via Selective Inter-client Transfer (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Continual Learning (CL) is a privacy-preserving machine learning technique that enables collaborative training of ML models by sharing model parameters across distributed clients. |
| Approach: | They propose a framework which selectively combines model parameters of foreign clients to maximize knowledge transfer while preserving privacy. |
| Outcome: | The proposed framework improves the performance of a text classification task using five datasets from diverse domains while preserving privacy. |
Similar Papers
FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Tasks (2022.findings-naacl)
Copied to clipboard
Bill Yuchen Lin, Chaoyang He, Zihang Ze, Hulin Wang, Yufen Hua, Christophe Dupuy, Rahul Gupta, Mahdi Soltanolkotabi, Xiang Ren, Salman Avestimehr
| Challenge: | Increasing concerns and regulations about data privacy necessitate the study of privacy-preserving, decentralized learning methods for natural language processing tasks. |
| Approach: | They propose a framework for evaluating federated learning methods on four different tasks . they propose federation between Transformer-based language models and FL methods . |
| Outcome: | The proposed framework compares FL methods on four different tasks under non-IID partitioning strategies. |
Continual Lifelong Learning in Natural Language Processing: A Survey (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to continual learning (CL) are costly and time-consuming. |
| Approach: | They propose to examine the problem of continual learning in NLP through the lens of various NLP tasks and provide a critical review of existing methods. |
| Outcome: | The proposed methods are critical to the development of CL models and provide a critical review of existing methods and datasets. |
Federated Learning for Semantic Parsing: Task Formulation, Evaluation Setup, New Algorithms (2023.acl-long)
Copied to clipboard
| Challenge: | Neural semantic parsers have achieved remarkable performance in recent years, but they are data-hungry and require annotators to have intimate knowledge of formal programs. |
| Approach: | They propose a task where multiple clients collaboratively train one global model without sharing their semantic parsing data. |
| Outcome: | The proposed model improves performance on three widely adopted FL algorithms (FedAvg, FedOPT and FedProx) and clients with smaller datasets enjoy faster performance. |
Coordinated Replay Sample Selection for Continual Federated Learning (2023.emnlp-industry)
Copied to clipboard
Jack Good, Jimit Majmudar, Christophe Dupuy, Jixuan Wang, Charith Peris, Clement Chung, Richard Zemel, Rahul Gupta
| Challenge: | Continual Federated Learning (CFL) combines decentralized learning with continuous learning . ubiquity of personal devices with a network connection offers rich source of data for learning problems . |
| Approach: | They propose to combine decentralized learning with a continuous learning approach . they propose to coordinate gradient-based replay sample selection across clients . |
| Outcome: | The proposed method shows gains early in the low replay size regime, when the budget for storing past data is small. |
Personalized Federated Learning for Text Classification with Gradient-Free Prompt Tuning (2024.findings-naacl)
Copied to clipboard
Rui Wang, Tong Yu, Ruiyi Zhang, Sungchul Kim, Ryan Rossi, Handong Zhao, Junda Wu, Subrata Mitra, Lina Yao, Ricardo Henao
| Challenge: | Pretrained language models (PLMs) are used for personalized federated learning . communication costs are high with large PLMs, and local training is expensive . |
| Approach: | They propose a framework for federated learning with pretrained language models . they propose 'discrete local search' and compression mechanism for local training . |
| Outcome: | The proposed framework achieves superior performance compared with baselines. |
Continual Learning for Text Classification with Information Disentanglement Based Regularization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing continual learning methods focus on preserving knowledge from previous tasks . Continual learning is a useful tool for learning over time, but it is not always possible to generalize to new tasks. |
| Approach: | They propose a disentanglement-based regularization method for continual learning on text classification that disentangles text hidden spaces into generic representations and regularizes them differently to constrain knowledge required to generalize. |
| Outcome: | The proposed method disentangles text hidden spaces into representations that are generic to all tasks and representations specific to each individual task. |
Exploring Cross-Client Memorization of Training Data in Large Language Models for Federated Learning (2026.acl-short)
Copied to clipboard
Tinnakit Udsa, Can Udomcharoenchaikit, Patomporn Payoungkhamdee, Sarana Nutanong, Norrathep Rattanavipanon
| Challenge: | Existing methods to assess memorization in federated learning focus on one sample at a time . centralized learning does not eliminate the risk of memorizing large language models . |
| Approach: | They propose a framework that quantifies both intra- and inter-client memorization in FL . they use fine-grained cross-sample memorisation measurement across all clients . |
| Outcome: | The proposed framework quantifies both intra- and inter-client memorization in FL using fine-grained cross-sample memorisation measurement across all clients. |
Dual Contrastive Learning Framework for Incremental Text Classification (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In incremental learning, large models learn and refresh knowledge continuously . many approaches have been proposed to preserve knowledge from previous tasks while learning new concepts in online NLP applications. |
| Approach: | They propose a dual contrastive learning framework that fosters transferability across different tasks . they use global contrastive and task-specific learning to promote a generalized embedding space . |
| Outcome: | The proposed framework outperforms the current state-of-the-art methods on text datasets. |
Multilingual Continual Learning using Attention Distillation (2025.coling-industry)
Copied to clipboard
| Challenge: | Existing models for Query-product relevance classification are not accurate across multiple languages. |
| Approach: | They propose a multilingual continual learning framework that adds adapters for each new language and incorporates a fusion layer above language-specific adapters. |
| Outcome: | The proposed approach reduces trainable parameters by 80% while outperforming SOTA CL methods on proprietary and external datasets. |
Safely Learning with Private Data: A Federated Learning Framework for Large Language Model (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing large language models (LLMs) use large amounts of public data and massive parameters, but private data is often stored in isolated data silos. |
| Approach: | They propose a Federated Learning framework for large language models which offloads most training parameters to the server while training embedding and output layers locally. |
| Outcome: | The proposed framework achieves comparable metrics to centralized chatGLM model on NLU and generation tasks. |