Simple Data Augmentation with the Mask Token Improves Domain Adaptation for Dialog Act Tagging (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on DA tagging focus on human-human social conversations, which is less applicable for task-oriented setting. |
| Approach: | They propose a controllable mechanism that augments text input by leveraging the pre-trained Mask token from BERT model. |
| Outcome: | The proposed mechanism augments text input by leveraging the pre-trained Mask token from BERT model. |
Similar Papers
Automatically Learning Data Augmentation Policies for Dialogue Tasks (D19-1)
Copied to clipboard
| Challenge: | Recent advances in automatic data augmentation have focused on computer vision tasks where it is easy to apply imperceptible perturbations without changing an image’s semantic meaning. |
| Approach: | They adapt AutoAugment to automatically discover effective perturbation policies for natural language processing (NLP) tasks such as dialogue generation. |
| Outcome: | The proposed algorithm reduces data-level model bias by using a controller trained on the target task. |
Data Augmentation and Learned Layer Aggregation for Improved Multilingual Language Understanding in Dialogue (2022.findings-acl)
Copied to clipboard
| Challenge: | Multi-SentAugment and LayerAgg are self-training methods that augment available training data with similar (automatically labelled) in-domain sentences from large monolingual Web-scale corpora. |
| Approach: | They propose to use multi-sentaugment and layeragg to improve dialogue natural language understanding across multiple languages. |
| Outcome: | The proposed methods generalise well in zero- and few-shot scenarios and leverage external unannotated data sources. |
ISO-Standard Domain-Independent Dialogue Act Tagging for Conversational Agents (C18-1)
Copied to clipboard
| Challenge: | Existing methods for DA annotation are incompatible with each other and do not cover all aspects necessary for open-domain human-machine interaction. |
| Approach: | They propose to map publicly available corpora to a subset of the ISO standard and create a task-independent training corpus for DA classification. |
| Outcome: | The proposed method can train a domain-independent DA tagger on out-of-domain conversational data and achieve robustness across different DA categories. |
AutoAugment Is What You Need: Enhancing Rule-based Augmentation Methods in Low-resource Regimes (2024.eacl-srw)
Copied to clipboard
| Challenge: | Existing methods for text data augmentation suffer from potential semantic damage due to the discrete nature of sentences. |
| Approach: | They propose to adapt AutoAugment to solve this problem by using softEDA to increase text data. |
| Outcome: | The proposed method can boost existing augmentation methods and enhance cutting-edge pretrained language models. |
N-Shot Learning for Augmenting Task-Oriented Dialogue State Tracking (2022.findings-acl)
Copied to clipboard
| Challenge: | augmentation of task-oriented dialogues has followed standard methods for plain-text despite its richly annotated structure. |
| Approach: | They propose an augmentation framework that utilizes belief state annotations to match turns from various dialogues and form new synthetic dialogues in a bottom-up manner. |
| Outcome: | The proposed framework performs better on seen values and more robust to unseen values on n-shot training scenarios. |
Semi-supervised Domain Adaptation for Dependency Parsing via Improved Contextualized Word Representations (2020.coling-main)
Copied to clipboard
| Challenge: | Recent advances in deep neural network models have improved parsing performance on in-domain texts . however, the problem is to improve performance on out-of-domain text data when there is only a small-scale out-domain labeled data. |
| Approach: | They propose to use adversarial learning and fine-tuning BERT to improve contextualized word representations on out-of-domain texts. |
| Outcome: | The proposed models achieve consistent improvement and fine-tune BERT processes boost parsing accuracy by a large margin. |
On-the-fly Denoising for Data Augmentation in Natural Language Understanding (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods to improve data augmentation performance may introduce noisy data that impairs training. |
| Approach: | They propose an on-the-fly denoising technique that learns from soft augmented labels provided by an organic teacher model trained on the cleaner original dataset. |
| Outcome: | The proposed method improves on text classification and question-answering tasks on general augmentation techniques and prevents overfitting on noisy labels. |
Neural Mask Generator: Learning to Generate Adaptive Word Maskings for Language Model Adaptation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train language models on diverse text corpora have brought up performance improvements on several natural language understanding (NLU) tasks. |
| Approach: | They propose a method to automatically generate domain- and task-adaptive maskings of a given text for self-supervised pre-training. |
| Outcome: | The proposed framework outperforms rule-based masking strategies on question answering and text classification datasets on which it outperformed rule-driven masking techniques. |
Explore More Guidance: A Task-aware Instruction Network for Sign Language Translation Enhanced with Data Augmentation (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies focus on the recognition step, while paying less attention to sign language translation. |
| Approach: | They propose a task-aware instruction network, namely TIN-SLT, for sign language translation, by introducing the isntruction module and the learning-based feature fuse strategy into a Transformer network. |
| Outcome: | The proposed system outperforms existing solutions on two benchmark datasets, PHOENIX-2014-T and ASLG-PC12, and outperformed previous best solutions by 1.65 and 1.42 in terms of BLEU-4. |
Self-training Improves Pre-training for Natural Language Understanding (2021.naacl-main)
Copied to clipboard
Jingfei Du, Edouard Grave, Beliz Gunel, Vishrav Chaudhary, Onur Celebi, Michael Auli, Veselin Stoyanov, Alexis Conneau
| Challenge: | Unsupervised pretraining has led to improvements in natural language understanding . a data augmentation method can be used to generate labels for unlabeled examples . |
| Approach: | They propose a semi-supervised method which uses unlabeled data to retrieve sentences from a database of billions of unlabed sentences crawled from the web. |
| Outcome: | The proposed method improves on standard text classification benchmarks by 2.6% and knowledge distillation by few shots. |