Multi-Source Text Classification for Multilingual Sentence Encoder with Machine Translation (2024.naacl-srw)
Copied to clipboard
| Challenge: | Pre-trained multilingual sentence encoders suffer from performance degradation for non-English languages. |
| Approach: | They propose a method of machine translating a source sentence into English and then inputting it together with the source sentence in a multi-source manner. |
| Outcome: | The proposed method improves the performance of pre-trained multilingual sentence encoders in Japanese on sentiment analysis and topic classification tasks. |
Similar Papers
A Simple and Effective Unified Encoder for Document-Level Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models for document-level machine translation use two separate encoders to model the source sentences and document- level contexts. |
| Approach: | They propose a unified encoder that can outperform existing models of dual-encoder models . they propose to use document-level contexts to model the interaction between the contexts and the source sentences . |
| Outcome: | The proposed model outperforms baseline models of dual-encoder models in terms of BLEU and METEOR scores. |
Does Multi-Encoder Help? A Case Study on Context-Aware Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | In encoder-decoder neural models, multiple encoders are used to represent contextual information in addition to the individual sentence. |
| Approach: | They propose to use multiple context encoders to encode the individual sentences in document-level neural machine translation (NMT) They propose a noisy dropout setup and a single-encoder approach to encode context sentences. |
| Outcome: | The proposed approach encodes the context and the current sentence without contexts. |
Multilingual Machine Translation: Closing the Gap between Shared and Language-specific Encoder-Decoders (2021.eacl-main)
Copied to clipboard
| Challenge: | State-of-the-art multilingual machine translation relies on a universal encoder-decoder, which requires retraining the entire system to add new languages. |
| Approach: | They propose an encoder-decoder approach that can be extended to new languages by learning their corresponding modules. |
| Outcome: | The proposed approach outperforms the universal encoder-decoder by 3.28 BLEU points on average while allowing to add new languages without retraining the rest of the modules. |
Improving Multilingual Neural Machine Translation with Auxiliary Source Languages (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work has shown that translating from multiple source languages improves translation quality. |
| Approach: | They propose to exploit multiple source sentences from auxiliary languages to improve multilingual translation in a more common scenario by using synthetic multi-source corpora. |
| Outcome: | Extensive experiments on Chinese/English-Japanese and a large-scale multilingual translation benchmark show that the proposed model outperforms the baseline model significantly by +4.0 BLEU. |
Multilingual Universal Sentence Encoder for Semantic Retrieval (2020.acl-demos)
Copied to clipboard
Yinfei Yang, Daniel Cer, Amin Ahmad, Mandy Guo, Jax Law, Noah Constant, Gustavo Hernandez Abrego, Steve Yuan, Chris Tar, Yun-hsuan Sung, Brian Strope, Ray Kurzweil
| Challenge: | Using a multi-task trained dual-encoder, our models embed text from 16 languages into a shared semantic space. |
| Approach: | They propose retrieval focused multilingual sentence embedding models on TensorFlow Hub. |
| Outcome: | The models achieve state-of-the-art on monolingual and cross-lingual retrieval (SR) and retrieval question answering (ReQA) competitive performance is obtained on related tasks of translation pair bitext retrieval and retrieving question answering. |
Improving Multilingual Neural Machine Translation by Utilizing Semantic and Linguistic Features (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing models do not differentiate between semantic and linguistic features, resulting in the entanglement of knowledge and linguistics within the model. |
| Approach: | They propose to exploit both semantic and linguistic features to enhance multilingual translation by disentangling encoder representations and integrating low-level linguistic encoders. |
| Outcome: | The proposed model improves zero-shot translation while maintaining performance in supervised translation on multilingual datasets. |
Context-Interactive Pre-Training for Document Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Document machine translation typically suffers from a lack of document-level bilingual data. |
| Approach: | They propose a document machine translation model that incorporates contextual information into the training signals by capturing cross-sentence dependency within the target document and cross sentence translation to make better use of contextual information. |
| Outcome: | The proposed model outperforms baselines on three benchmark datasets and significantly outperformed previous approaches. |
Sentence Compression for Arbitrary Languages via Multilingual Pivoting (D18-1)
Copied to clipboard
| Challenge: | a new study advocates the use of bilingual corpora for sentence compression models . previous work focused on word deletion, while others view sentence compression as a general text rewriting problem. |
| Approach: | They propose to use bilingual corpora for training sentence compression models. |
| Outcome: | The proposed model can be trained for any language as long as a bilingual corpus is available . it performs arbitrary rewrites without access to compression specific data . |
A Semi-supervised Approach to Generate the Code-Mixed Text using Pre-trained Encoder and Transfer Learning (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to train neural network-based models for code-mixing are limited due to language specificity of code-mixed text. |
| Approach: | They propose a deep learning approach to generate code-mixed text from English to multiple languages without any parallel data. |
| Outcome: | The proposed approach generates a code-mixed text from English to multiple languages without any parallel data. |
Modular Sentence Encoders: Separating Language Specialization from Cross-Lingual Alignment (2025.acl-long)
Copied to clipboard
| Challenge: | Multilingual sentence encoders are often trained to map sentences from different languages into a shared semantic vector space . cross-lingual alignment training distorts optimal monolingual structure of semantic spaces of individual languages . a modular solution can be used for cross-linguistic tasks such as cross-language semantic similarity and zero-shot transfer . |
| Approach: | They propose a modular training system that embeds sentences from different languages into a shared semantic vector space. |
| Outcome: | The proposed solution achieves better performance across all tasks compared to monolithic models. |