Papers with IID
FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Tasks (2022.findings-naacl)
Copied to clipboard
Bill Yuchen Lin, Chaoyang He, Zihang Ze, Hulin Wang, Yufen Hua, Christophe Dupuy, Rahul Gupta, Mahdi Soltanolkotabi, Xiang Ren, Salman Avestimehr
| Challenge: | Increasing concerns and regulations about data privacy necessitate the study of privacy-preserving, decentralized learning methods for natural language processing tasks. |
| Approach: | They propose a framework for evaluating federated learning methods on four different tasks . they propose federation between Transformer-based language models and FL methods . |
| Outcome: | The proposed framework compares FL methods on four different tasks under non-IID partitioning strategies. |
Pretrained Models for Multilingual Federated Learning (2022.naacl-main)
Copied to clipboard
| Challenge: | Federated Learning (FL) is a machine learning technique that trains a model across multiple distributed clients holding local data samples, without ever storing client data in a central location. |
| Approach: | They propose to use pretrained models to study three multilingual language tasks . they also examine impact of non-IID text on FL in naturally occurring data . |
| Outcome: | The proposed methods perform better than centralized learning even when using non-IID partitioning. |
Investigating Selective Prediction Approaches Across Several Tasks in IID, OOD, and Adversarial Settings (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to selective prediction are not consistently outperform the simplest baseline MaxProb in all three settings. |
| Approach: | They propose to use a large-scale setup of 17 datasets to study selective prediction in NLP tasks using in-domain, out-of-domain and adversarial settings. |
| Outcome: | The proposed approaches outperform the simplest baseline MaxProb in in-domain, out-of-domain and adversarial settings, but none consistently outperformed in all three settings. |
Structurally Diverse Sampling for Sample-Efficient Training and Comprehensive Evaluation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to generalize compositionally are inadequate, but there is no evidence for this. |
| Approach: | They propose a model-agnostic algorithm for subsampling instances with diverse structures from a labeled instance pool with structured outputs. |
| Outcome: | The proposed algorithm leads to comparable or better generalization than prior algorithms in 9 out of 10 dataset-split type pairs. |
Generating Diverse Hypotheses for Inductive Reasoning (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent studies suggest that large language models (LLMs) can engage in inductive reasoning by sampling multiple hypotheses about the rules and selecting the one that best explains the observations. |
| Approach: | They propose to increase the temperature parameter to enhance diversity by sampling multiple hypotheses and selecting the one that best explains the observations. |
| Outcome: | The proposed method improves diversity while maintaining text quality while increasing temperature. |
Federated Document-Level Biomedical Relation Extraction with Localized Context Contrast (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies on relation extraction focus on document-level training without sharing raw medical texts. |
| Approach: | They propose a federated framework for relation extraction that enables collaborative training without sharing raw medical texts. |
| Outcome: | The proposed framework extends document-level relation extraction to a federated environment. |
Counterfactual Active Learning for Out-of-Distribution Generalization (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies on active learning methods focus on the out-of-distribution generalization of out- of-distortion samples. |
| Approach: | They propose a counterfactual active learning approach that empowers active learning with counterfact thinking to bridge the seen samples with unseen cases. |
| Outcome: | The proposed approach outperforms existing active learning methods on public datasets with comparable IID performance. |
KoRC: Knowledge Oriented Reading Comprehension Benchmark for Deep Text Understanding (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for deep text understanding have encountered two major limitations . most require human annotation of knowledge, which leads to limited knowledge coverage . |
| Approach: | They propose a benchmark to help readers understand a document with prior knowledge . they use massive knowledge bases to guide annotators and large language models to construct knowledgable questions . |
| Outcome: | The proposed benchmarks have limited knowledge coverage and use choices or spans as answers, which results in narrow answer space. |
Unveiling Project-Specific Bias in Neural Code Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) based neural code models struggle to generalize effectively to real-world inter-project out-of-distribution data. |
| Approach: | They propose a Cond-Idf measurement to measure the relatedness of a token with a label and its project-specificness. |
| Outcome: | The proposed framework improves both inter-project OOD generalization and adversarial robustness while not sacrificing accuracy on intra-project IID data. |