| Challenge: | In structured prediction problems, labeling an input string with a length-T sequence of tags becomes intractable. |
| Approach: | They propose a sequential Monte Carlo method for sampling annotations of an input string from a given probability model. |
| Outcome: | The proposed method improves the quality of the sample. |
Similar Papers
Towards a Better Understanding of Label Smoothing in Neural Machine Translation (2020.aacl-main)
Copied to clipboard
| Challenge: | In recent years, Neural Network (NN) models bring steady and concrete improvements on the task of Machine Translation (MT). |
| Approach: | They propose to penalize over-confident outputs and regularize the model so that its outputs do not diverge too much from some prior distribution. |
| Outcome: | The proposed method is well-motivated and can improve the performance of strong neural machine translation systems. |
Text Smoothing: Enhance Various Data Augmentation Methods on Text Classification Tasks (2022.acl-short)
Copied to clipboard
| Challenge: | Experimental results show text smoothing outperforms data augmentation methods by a substantial margin. |
| Approach: | They propose to use a masked language model to convert a token to a smoothed representation by converting a sentence from its one-hot representation to 'controllable smoothes' they propose to combine text smoothing with other data augmentation methods to achieve better performance. |
| Outcome: | The proposed method outperforms mainstream data augmentation methods by a substantial margin on different datasets in a low-resource regime. |
Unifying Input and Output Smoothing in Neural Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Recent methods that smooth input and output of neural machine translation systems bring significant improvements in performance. |
| Approach: | They propose a method that replaces one-hot representations with soft posterior distributions of an external language model, smoothing the input of machine translation systems. |
| Outcome: | The proposed method improves translation performance on small datasets and larger datasets. |
Incorporating Exponential Smoothing into MLP: a Simple but Effective Sequence Model (2024.findings-naacl)
Copied to clipboard
| Challenge: | Structured State Space models (SSMs) have been used for long-range sequence learning but are limited in their complexity and computational and memory requirements. |
| Approach: | They propose to incorporate a simple SSM into an element-wise MLP to reduce inductive bias. |
| Outcome: | The proposed model achieves comparable results to existing models on the LRA benchmark. |
Differentiable Sampling with Flexible Reference Word Order for Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Existing approaches to correct exposure bias in machine translation are inadequate . scheduled sampling assumes that words are aligned at each time step . |
| Approach: | They propose a differentiable sampling algorithm that optimizes the probability that the reference can be aligned with the sampled output. |
| Outcome: | The proposed approach improves BLEU on translation tasks and is simpler to train with no sampling schedule. |
The Role of n-gram Smoothing in the Age of Neural Networks (2024.naacl-long)
Copied to clipboard
| Challenge: | n-gram smoothing techniques were used to overcome overfitting problems in neural language models for decades. |
| Approach: | They propose to convert any n-gram smoothing technique into a regularizer compatible with neural language models. |
| Outcome: | The proposed regularizers outperform label smoothing on language modeling and machine translation. |
Training for Gibbs Sampling on Conditional Random Fields with Neural Scoring Factors (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in NLP focus on simple approaches to model the output label space . graphical models are often limited to (heuristic) greedy search and its variants . |
| Approach: | They propose an approach for efficiently training and decoding hybrids of graphical and graphical models based on Gibbs sampling. |
| Outcome: | The proposed approach improves on Dutch and Dutch with graphical models . the proposed model improves over a strong baseline on three languages . |
A Detailed Evaluation of Neural Sequence-to-Sequence Models for In-domain and Cross-domain Text Simplification (L18-1)
Copied to clipboard
| Challenge: | Xu et al., 2016) show that a simple neural architecture can be efficiently used for in-domain and cross-domain text simplification. |
| Approach: | They evaluate neural sequence-to-sequence models for text simplification on Wikipedia and Newsela datasets. |
| Outcome: | The proposed model can generalize across corpora and overcome challenges when tested on Wikipedia and Newsela datasets. |
Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement (D18-1)
Copied to clipboard
| Challenge: | Despite its success, neural autoregressive modeling has its weakness in decoding, i.e., finding the most likely sequence. |
| Approach: | They propose a conditional non-autoregressive neural sequence model based on iterative refinement based upon latent variable models and conditional denoising autoencoders. |
| Outcome: | The proposed model significantly speeds up decoding while maintaining the generation quality comparable to the autoregressive counterpart. |
Sampling and Filtering of Neural Machine Translation Distillation Data (2021.naacl-srw)
Copied to clipboard
| Challenge: | In most of neural machine translation distillation or stealing scenarios, the highest-scoring hypothesis of the target model is used to train a new model. |
| Approach: | They propose to use the highest-scoring hypothesis of the target model (teacher) to train a new model (student). |
| Outcome: | The proposed method improves the performance of MT models in English to Czech and with reference translations. |