Pseudo-Bidirectional Decoding for Local Sequence Transduction (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Local sequence transduction tasks involve massive overlapping between source and target sequences . experimental results show that Pseudo-Bidirectional Decoding improves performance of standard seq2seq models. |
| Approach: | They propose a simple but versatile approach for local sequence transduction tasks . they propose to copy source tokens to decoder as pseudo future context . |
| Outcome: | The proposed approach improves the performance of standard seq2seq models on LST tasks. |
Similar Papers
A Framework for Bidirectional Decoding: Case Study in Morphological Inflection (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing encoder-decoders that generate sequences from left to right are prone to errors due to the "snowballing" effect. |
| Approach: | They propose a transformer-based encoder-decoder framework that produces sequences from the "outside-in" they argue that this approach is more principled than prior bidirectional decoders . |
| Outcome: | The proposed model beats the current system by over 4.7 and 2.7 points in accuracy on 2022 and 2023 tasks. |
Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | End-to-end speech translation models can be trained to leverage source text . however, since the input modalities are different, it is difficult to leverage the source text successfully. |
| Approach: | They propose to leverage source transcriptions via pre-training and joint training with ASR and NMT tasks. |
| Outcome: | The proposed model predicts paraphrased transcriptions as an auxiliary task with a single decoder. |
SepSeq: A Training-Free Framework for Long Numerical Sequence Processing in LLMs (2026.findings-acl)
Copied to clipboard
Jie Sun, Yu Liu, Lu Han, Qiwen Deng, Xiang Shu, Yang Xiao, Lintao Ma, Xingyu Lu, Jun Zhou, Pengfei Liu, Jiancan Wu, Xiang Wang
| Challenge: | Existing large-scale large-context models suffer from performance degradation when processing long numerical sequences. |
| Approach: | They propose a framework to mitigate attention dispersion by strategically inserting separator tokens into the model to recalibrat attention to local segments while preserving global context. |
| Outcome: | The proposed framework improves accuracy and reduces inference token consumption by 16.4% on 9 widely-adopted LLMs. |
Dynamic Attention-Guided Context Decoding for Mitigating Context Faithfulness Hallucinations in Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods, such as a n-terminal coding, do not provide accurate data for large language models. |
| Approach: | They propose a lightweight framework that leverages attention distributions and uncertainty signals in a single-pass decoding. |
| Outcome: | Experiments on open-book QA datasets show that DAGCD improves faithfulness and robustness while preserving computational efficiency. |
Local String Transduction as Sequence Labeling (C18-1)
Copied to clipboard
| Challenge: | String transduction and sequence labeling are often treated as separate entities and often give treatment to different problems in NLP. |
| Approach: | They propose to reduce string transduction to sequence labeling by using a finite-state technique that uses string transducing and sequence labelling. |
| Outcome: | The proposed method performs better than seq2seq models and yields state-of-the-art results in several cases. |
Bidirectional Transformer Reranker for Grammatical Error Correction (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained seq2seq models suffer from a prediction bias due to their unidirectional decoding. |
| Approach: | They propose a bidirectional Transformer reranker that re-estimates the probability of each candidate sentence generated by pre-trained seq2seq models. |
| Outcome: | The proposed model improves on the original model and gives a 59.52 GLEU score on the JFLEG corpus. |
DR-BiLSTM: Dependent Reading Bidirectional LSTM for Natural Language Inference (N18-1)
Copied to clipboard
Reza Ghaeini, Sadid A. Hasan, Vivek Datla, Joey Liu, Kathy Lee, Ashequl Qadir, Yuan Ling, Aaditya Prakash, Xiaoli Fern, Oladimeji Farri
| Challenge: | Existing approaches to natural language inference rely on simple reading mechanisms for independent encoding of the premise and hypothesis. |
| Approach: | They propose a novel bidirectional dependent reading network to efficiently model the relationship between a premise and a hypothesis during encoding and inference. |
| Outcome: | The proposed model outperforms existing methods by a considerable margin on the Stanford Natural Language Inference (SNLI) dataset. |
Attending to Future Tokens for Bidirectional Sequence Generation (D19-1)
Copied to clipboard
| Challenge: | Neural sequence generation is typically performed token-by-token and left-to-right. |
| Approach: | They propose to use placeholder tokens to make the sequence generation process bidirectional by taking past and future tokens into consideration when generating the actual output token. |
| Outcome: | The proposed approach outperforms baselines on two conversational tasks by a large margin. |
Exact Hard Monotonic Attention for Character-Level Transduction (P19-1)
Copied to clipboard
| Challenge: | Neural sequence-to-sequence models with soft attention outperform monotonic models . current dominant method is the neural sequenceto-Sequency model with soft focus . |
| Approach: | They develop a hard attention sequence-to-sequence model that enforces strict monotonicity and learns alignment jointly. |
| Outcome: | The proposed model achieves state-of-the-art on grapheme-to-phoneme conversion and morphological inflection generation. |
Deconvolution-Based Global Decoding for Neural Machine Translation (C18-1)
Copied to clipboard
| Challenge: | Existing models for Neural Machine Translation (NMT) use Recurrent Neural Network (RNN) to generate translation word by word following a sequential order. |
| Approach: | They propose a Neural Machine Translation (NMT) model that decodes the sequence with the guidance of its structural prediction of the target-side context. |
| Outcome: | The proposed model is more competitive compared with the state-of-the-art methods and reduces repetition with the instruction from the target-side context for decoding. |