Automatic Rule Induction for Efficient Semi-Supervised Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to generalize from labeled and unlabeled data are difficult to explain and behave unreliably. |
| Approach: | They propose a framework for automatic discovery and integration of symbolic rules into pretrained transformer models by using an attention mechanism. |
| Outcome: | The proposed framework can improve state-of-the-art methods with no manual effort and minimal computational overhead. |
Similar Papers
Self-Training with Weak Supervision (2021.naacl-main)
Copied to clipboard
| Challenge: | State-of-the-art deep neural networks require large amounts of labeled training data that is expensive to obtain or not available for many tasks. |
| Approach: | They propose a weak supervision framework that leverages all available data for a given task . they leverage task-specific unlabeled data through self-training with a model that predicts pseudo-labels for instances that may not be covered by weak rules . |
| Outcome: | The proposed framework improves on state-of-the-art datasets on six benchmark tasks. |
CoAug: Combining Augmentation of Labels and Labelling Rules (2023.findings-acl)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) tasks require large labeled datasets to perform well. |
| Approach: | They propose a co-augmentation framework that bootstraps predictions from each model to improve few-shot models and rule-augmentation models by bootstrapping them. |
| Outcome: | The proposed model outperforms strong weak-supervision-based models by 6.5 F1 points . the proposed model can learn from limited labeled data and perform better on small datasets . |
Reasoning Makes Good Annotators : An Automatic Task-specific Rules Distilling Framework for Low-resource Relation Extraction (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to extract knowledge from unlabeled data generate noise labels. |
| Approach: | They propose an automatic task-specific rules distilling framework to generate a logic rule from unlabeled data. |
| Outcome: | The proposed framework could power the labeling ability by discovering reliable model-labeled data. |
AutoAugment Is What You Need: Enhancing Rule-based Augmentation Methods in Low-resource Regimes (2024.eacl-srw)
Copied to clipboard
| Challenge: | Existing methods for text data augmentation suffer from potential semantic damage due to the discrete nature of sentences. |
| Approach: | They propose to adapt AutoAugment to solve this problem by using softEDA to increase text data. |
| Outcome: | The proposed method can boost existing augmentation methods and enhance cutting-edge pretrained language models. |
ARISE: Iterative Rule Induction and Synthetic Data Generation for Text Classification (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing frameworks for large language models (LLMs) generate high-quality synthetic data that can be used to supplement training data or surpass crowd-sourced annotations. |
| Approach: | They propose a framework that iteratively induces rules and generates synthetic data for text classification. |
| Outcome: | The proposed framework outperforms existing models on in-context learning and fine-tuning settings by using augmented data. |
GLaRA: Graph-based Labeling Rule Augmentation for Weakly Supervised Named Entity Recognition (2021.eacl-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) models often need to be trained with many manual labels to perform well. |
| Approach: | They propose to train named entity recognition systems using heuristic labeling rules . they create a graph with nodes representing candidate rules extracted from unlabeled data . |
| Outcome: | The proposed method achieves an average improvement of +20% over the baseline on three NER datasets. |
From Examples to Rules: Neural Guided Rule Synthesis for Information Extraction (2022.lrec-1)
Copied to clipboard
Robert Vacareanu, Marco A. Valenzuela-Escárcega, George Caique Gouveia Barbosa, Rebecca Sharp, Gustave Hahn-Powell, Mihai Surdeanu
| Challenge: | a "deep learning tsunami" has brought tremendous improvements in performance to most NLP applications. |
| Approach: | They propose a method for rule synthesis from examples that combines the advantages of deep learning and rule-based methods. |
| Outcome: | The proposed method achieves state-of-the-art on 1-shot task and competitive performance in 5-shot scenario. |
Leveraging Grammar Induction for Language Understanding and Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing grammar induction methods do not provide sufficient performance in downstream tasks. |
| Approach: | They propose an unsupervised grammar induction method for language understanding and generation using a grammar parser and a syntactic mask. |
| Outcome: | The proposed method performs better on from-scratch and pre-trained scenarios. |
Do Syntax Trees Help Pre-trained Transformers Extract Information? (2021.eacl-main)
Copied to clipboard
| Challenge: | Recent work suggests that incorporating syntax information from dependency trees can improve task-specific transformer models. |
| Approach: | They propose to incorporate dependency tree information into pre-trained transformers for three tasks . they propose a late fusion approach and a joint fusion technique to infuses syntax structure into attention layers. |
| Outcome: | The proposed models obtain state-of-the-art results on SRL and relation extraction tasks. |
Syntax-Enhanced Pre-trained Model (2021.acl-long)
Copied to clipboard
Zenan Xu, Daya Guo, Duyu Tang, Qinliang Su, Linjun Shou, Ming Gong, Wanjun Zhong, Xiaojun Quan, Daxin Jiang, Nan Duan
| Challenge: | Existing methods that use syntax of text in pre-training and fine-tuning suffer from discrepancy between the two stages. |
| Approach: | They propose a model that utilizes the syntactic structure of text in pre-training and fine-tuning stages. |
| Outcome: | The proposed model achieves state-of-the-art on six public benchmark datasets. |