| Challenge: | Existing bidirectional encoders require a restart when a new token is received. |
| Approach: | They propose a Hybrid Encoder with Adaptive Restart that enables asynchronous encoding of a new token in an incremental streaming input. |
| Outcome: | The proposed encoder offers FLOP savings in streaming settings up to 71.1% and outperforms bidirectional encoders for streaming predictions by up to +0% streaming exact match. |
Similar Papers
A Streaming Approach For Efficient Batched Beam Search (2020.emnlp-main)
Copied to clipboard
| Challenge: | During decoding, candidates terminate or are pruned according to heuristics, a streaming method is used to "refill" the batch after it finishes translating some fraction of the current batch. |
| Approach: | They propose an efficient batching strategy for variable-length decoding on GPU architectures by streamlining the batching process. |
| Outcome: | The proposed method reduces runtime by 71% compared to a fixed-width beam search baseline and 17% compared with a variable-widness baseline while matching baselines’ BLEU. |
On Sparsifying Encoder Outputs in Sequence-to-Sequence Models (2021.findings-acl)
Copied to clipboard
| Challenge: | Using sequence-to-sequence models, encoder outputs are usually transferred to the decoder for generation, but in this study, encoded outputs can be compressed to shorten the sequence for decoding. |
| Approach: | They propose to use a stochastic gate-based algorithm to mask encoder outputs to shorten the sequence delivered for decoding. |
| Outcome: | The proposed model can be used to shorten encoder outputs to short a sequence . the proposed model yields a speedup of up to 1.65 on document summarization and 1.20 on character-based machine translation tasks. |
wav2vec-S: Adapting Pre-trained Speech Models for Streaming (2024.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained speech models have advanced speech-related tasks, including speech recognition and translation. |
| Approach: | They propose a pre-trained speech model that incorporates modifications to ensure consistent speech representations during training and inference phases for streaming speech inputs. |
| Outcome: | The proposed model outperforms baseline models on speech recognition and translation tasks and achieves a superior balance between quality and latency. |
Who Needs Decoders? Efficient Estimation of Sequence-Level Attributes with Proxies (2024.eacl-long)
Copied to clipboard
| Challenge: | Autoregressive decoding is expensive for many sequence-to-sequence tasks, but for some downstream tasks, the actual decoding output is not needed, just attributes of the sequence. |
| Approach: | They propose non-autoregressive proxy models that can efficiently predict scalar-valued sequence-level attributes from the encodings, avoiding the expensive decoding stage. |
| Outcome: | The proposed models outperform ensembles in machine translation (MT) and automatic speech recognition (ASR) while being significantly faster. |
Incomplete Utterance Rewriting as Sequential Greedy Tagging (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies show that users of dialogue systems tend to use incomplete utterances which usually omit (a.k.a. ellipsis) or refer back (a k.k a co-reference) to the concepts that appeared in previous dialogue contexts. |
| Approach: | They propose a sequence tagging-based model which extracts information from context and introduce speaker-aware embedding to model speaker variation. |
| Outcome: | The proposed model outperforms previous models on restoration scores while having other metric scores comparable to state-of-the-art models. |
Bootstrapping a Music Voice Assistant with Weak Supervision (2021.naacl-industry)
Copied to clipboard
| Challenge: | Music listening is among the top-5 reasons of daily usage of voice assistants in the US. |
| Approach: | They propose a weakly-supervised method to label large amounts of voice query logs . they show that slot tagging models outperform models trained on hand-annotated or synthetic data . |
| Outcome: | The proposed method outperforms models trained on hand-annotated or synthetic data at a lower cost. |
When Generative Adversarial Networks Meet Sequence Labeling Challenges (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches for sequence labeling use a feature extractor and sequence tagger . a recent study shows that SLGAN is versatile and highly effective . |
| Approach: | They propose a framework that harnesses the capabilities of Generative Adversarial Networks to address sequence labeling challenges. |
| Outcome: | The proposed framework exhibits strong adaptability to various sequence labeling tasks. |
Unleashing the True Potential of Sequence-to-Sequence Models for Sequence Tagging and Structure Parsing (2023.tacl-1)
Copied to clipboard
| Challenge: | Sequence-to-Sequence (S2S) models have been successful on text generation tasks . however, learning complex structures with S2S models remains challenging . |
| Approach: | They propose to use constrained decoding to model part-of-speech tagging, named entity recognition, constituency, and dependency parsing tasks with 3 lexically diverse linearization schemas and corresponding constrained coding methods. |
| Outcome: | The proposed methods outperform the state-of-the-art on four core tasks. |
Encode, Tag, Realize: High-Precision Text Editing (D19-1)
Copied to clipboard
| Challenge: | Neural sequence-to-sequence models provide a powerful framework for learning to translate source texts into target texts. |
| Approach: | They propose a sequence tagging approach that casts text generation as a text editing task. |
| Outcome: | The proposed model outperforms strong seq2seq models on sentence fusion, sentence splitting, abstractive summarization, and grammar correction tasks and achieves state-of-the-art performance. |
Teaching BERT to Wait: Balancing Accuracy and Latency for Streaming Disfluency Detection (2022.naacl-main)
Copied to clipboard
| Challenge: | a recent study shows that current NLP models operate non-incrementally, causing unacceptable delays for the user. |
| Approach: | They propose a streaming BERT-based sequence tagging model that detects disfluencies in real-time . they train the model to decide whether to immediately output a prediction or wait for further context . |
| Outcome: | The proposed model produces accurate predictions sooner than baselines, with lower flicker . disfluencies hurt readability of ASR transcripts, erode model performance on downstream tasks . |