| Challenge: | Recent studies on disfluency detection heavily relies on human annotations, which are difficult and expensive to obtain in practice. |
| Approach: | They propose an unsupervised method that reweights the importance of each training example according to its grammatical feature and prediction confidence. |
| Outcome: | The proposed method improves 2.3 points over the current SOTA unsupervised method and is competitive with the SOTA supervised method. |
Similar Papers
Combining Self-Training and Self-Supervised Learning for Unsupervised Disfluency Detection (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to disfluency detection rely on human annotations, which are expensive to obtain. |
| Approach: | They propose an unsupervised learning paradigm which can work with unlabeled text corpora. |
| Outcome: | The proposed method performs better than existing supervised systems using word embeddings. |
Improving Disfluency Detection by Self-Training a Self-Attentive Model (2020.acl-main)
Copied to clipboard
| Challenge: | Existing self-attentive parsers using contextualized word embeddings produce state-of-the-art results in joint parsing and disfluency detection. |
| Approach: | They propose to use contextualized word embeddings to train a neural model using unlabeled data to train parsers. |
| Outcome: | The proposed method produces state-of-the-art results in parsing and disfluency detection in speech transcripts. |
Semi-Supervised Disfluency Detection (C18-1)
Copied to clipboard
| Challenge: | Detecting disfluency can be difficult because of the flexible nature of reparandum structure and the lack of a nested structure. |
| Approach: | They propose a semi-supervised approach which extracts hidden features from self-attention without any Recurrent Neural Network (RNN) or Convolutional Neural Net (CNN). |
| Outcome: | The proposed approach improves over baselines by using unlabelled data . identifying and removing non-fluent factors would help to improve spontaneous speech quality . |
Self-Discriminative Learning for Unsupervised Document Embedding (N19-1)
Copied to clipboard
| Challenge: | Existing methods for document embedding learning do not consider inter-document relationships. |
| Approach: | They propose to exploit the inter-document information and directly model the relations of documents in embedding space with a discriminative network and a novel objective. |
| Outcome: | The proposed method has errors that are 5 to 13% lower than state-of-the-art models and is even more pronounced in scarce label setting. |
LARD: Large-scale Artificial Disfluency Generation (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing datasets suffer from class imbalance issues, causing performance problems . Disfluency detection is a critical task in real-time dialogue systems . |
| Approach: | They propose a method for generating complex and realistic artificial disfluencies with little effort using a large-scale dataset. |
| Outcome: | The proposed method can handle repetitions, replacements, and restarts on a large-scale dataset with disfluencies. |
Self-Training for Unsupervised Neural Machine Translation in Unbalanced Training Data Scenarios (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods that use monolingual corpora for translation are not suitable for low-resource languages such as Estonian. |
| Approach: | They propose unsupervised neural machine translation (UNMT) that relies on monolingual corpora to train a robust UNMT system and improve its performance. |
| Outcome: | The proposed methods outperform conventional UNMT systems on several language pairs. |
Disfluency Detection using Auto-Correlational Neural Networks (D18-1)
Copied to clipboard
| Challenge: | a recent study proposes an auto-correlational neural network (ACNN) that can detect disfluency in speech . the model uses a convolutional neural system and augments it with a new auto-corrector . |
| Approach: | They propose a convolutional neural network model that captures "rough copy" dependencies . the model is based on a new auto-correlation operator that capture the kinds of "rough copies" dependency . |
| Outcome: | The proposed model outperforms the baseline CNN on a disfluency detection task with a 5% increase in f-score. |
Unsupervised Dense Retrieval with Relevance-Aware Contrastive Pre-Training (2023.findings-acl)
Copied to clipboard
| Challenge: | Dense retrievers have impressive performance, but their demand for abundant training data limits their application scenarios. |
| Approach: | They propose a method which uses unlabeled data to construct pseudo-positive examples from unlabelled data and then contrastively weighs the contrastive loss of different pairs according to the estimated relevance. |
| Outcome: | The proposed method beats the SOTA unsupervised Contriever model on BEIR and open-domain QA retrieval benchmarks and is a good few-shot learner. |
Self-training Improves Pre-training for Natural Language Understanding (2021.naacl-main)
Copied to clipboard
Jingfei Du, Edouard Grave, Beliz Gunel, Vishrav Chaudhary, Onur Celebi, Michael Auli, Veselin Stoyanov, Alexis Conneau
| Challenge: | Unsupervised pretraining has led to improvements in natural language understanding . a data augmentation method can be used to generate labels for unlabeled examples . |
| Approach: | They propose a semi-supervised method which uses unlabeled data to retrieve sentences from a database of billions of unlabed sentences crawled from the web. |
| Outcome: | The proposed method improves on standard text classification benchmarks by 2.6% and knowledge distillation by few shots. |
A Comparative Analysis of Unsupervised Language Adaptation Methods (D19-61)
Copied to clipboard
| Challenge: | Recent proposed approaches to perform unsupervised language adaptation lack annotated resources in less-resourced languages. |
| Approach: | They propose to use Adversarial Training, Sentence Encoder Alignment and Shared-Private Architecture to perform unsupervised language adaptation without using aligned sentences. |
| Outcome: | The proposed approaches are more suitable when the source and target language datasets contain other variations in content besides the language shift. |