The Lazy Encoder: A Fine-Grained Analysis of the Role of Morphology in Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural sequence-to-sequence models have proven effective for machine translation, but at the expense of interpretability. |
| Approach: | They analyze how morphological features are captured at different levels of the NMT encoder while varying the target language. |
| Outcome: | The proposed model is not interpretable, but only captures morphological features in context and only to the extent they are directly transferable to the target words. |
Similar Papers
A Morphology-Based Investigation of Positional Encodings (2024.emnlp-main)
Copied to clipboard
| Challenge: | Contemporary deep learning models handle languages with diverse morphology . morphological complexity of languages is closely linked with positional encodings . |
| Approach: | They propose to use positional encodings to integrate morphological complexity into deep learning models. |
| Outcome: | The proposed model improves on 22 languages and 5 downstream tasks. |
Low-resource neural machine translation with morphological modeling (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for character-based and sub-word tokenization are limited to the surface forms of the words. |
| Approach: | They propose a framework-solution for modeling complex morphology in low-resource settings using a transformer architecture and beam search-based decoder. |
| Outcome: | The proposed model improves translation performance on Kinyarwanda English translation using public-domain parallel text. |
Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translation (2025.findings-acl)
Copied to clipboard
Yingfeng Luo, Tong Zheng, Yongyu Mu, Bei Li, Qinghong Zhang, Yongqi Gao, Ziqiang Xu, Peinan Feng, Xiaoqian Liu, Tong Xiao, JingBo Zhu
| Challenge: | Recent advances in machine translation have focused on a single pre-trained decoder . encoder-decoder architectures have received relatively little attention in NMT . |
| Approach: | They propose a method that leverages LLMs as MT encoders and pairs them with lightweight decoders to develop universal translation models. |
| Outcome: | The proposed method matches or surpasses baselines in terms of translation quality but achieves 75% reduction in memory footprint of the KV cache. |
One Size Does Not Fit All: Comparing NMT Representations of Different Granularities (N19-1)
Copied to clipboard
| Challenge: | Recent work has shown that contextualized word representations are a viable alternative to simple word prediction tasks. |
| Approach: | They propose to use subword units and characters to model morphology, syntax, and semantics instead of word embeddings. |
| Outcome: | The proposed representations are better for modeling syntax and more robust to noisy input. |
Improving Character-Based Decoding Using Target-Side Morphological Information for Neural Machine Translation (N18-1)
Copied to clipboard
| Challenge: | Morphologically complex words (MCWs) are multi-layer structures consisting of different subunits, each of which carries semantic information and has a specific syntactic role. |
| Approach: | They propose an extension to the state-of-the-art model which works at the character level and boosts the decoder with target-side morphological information. |
| Outcome: | The proposed model improves on the state-of-the-art model and can be extended to include morphologically complex words (MCWs) in three languages. |
Proceedings of the 3rd Workshop on Neural Generation and Translation (D19-56)
Copied to clipboard
| Challenge: | The third workshop on neural generation and translation is held in london . the workshop received 68 submissions from leading minds in the field . |
| Approach: | the third workshop on neural generation and translation is held in london . the workshop will feature four invited talks from leading minds in the field . |
| Outcome: | the third workshop on neural generation and translation is held in london . the conference received 68 submissions from which 36 accepted . |
Does Multi-Encoder Help? A Case Study on Context-Aware Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | In encoder-decoder neural models, multiple encoders are used to represent contextual information in addition to the individual sentence. |
| Approach: | They propose to use multiple context encoders to encode the individual sentences in document-level neural machine translation (NMT) They propose a noisy dropout setup and a single-encoder approach to encode context sentences. |
| Outcome: | The proposed approach encodes the context and the current sentence without contexts. |
On the differences between BERT and MT encoder spaces and how to address them in translation tasks (2021.acl-srw)
Copied to clipboard
| Challenge: | Various studies show that pretrained language models cannot replace encoders in neural machine translation despite their success in other tasks. |
| Approach: | They propose a supervised transformation from one into the other to improve the applicability of BERT in neural machine translation. |
| Outcome: | The proposed transformations show that they cannot replace encoders in MT despite their success in other tasks. |
Towards Modeling the Style of Translators in Neural Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | a key ingredient of neural machine translation is the use of large datasets with different but consistent translation styles . however, the models do not capture the variety of translators' styles from the data . a recent study shows that style-augmented models can capture the style variations of translator . |
| Approach: | They propose to augment a neural machine translation model with translator information . they use TED talk datasets to model and control translator-related stylistic variations . |
| Outcome: | The proposed models capture the style variations of translators and generate translations with different styles on new data. |
Training and Adapting Multilingual NMT for Less-resourced and Morphologically Rich Languages (L18-1)
Copied to clipboard
| Challenge: | Using multilingual and multi-way neural machine translation approaches is a major advantage . training NMT systems for individual language pairs takes significantly more time than training of SMT systems . |
| Approach: | They propose to employ multilingual and multi-way neural machine translation approaches for morphologically rich languages such as Estonian and Russian. |
| Outcome: | The proposed approach improves translation quality by +3.27 BLEU points over baseline models. |