BACK-TRANSLATION

ExtraPhrase: Efficient Data Augmentation for Abstractive Summarization (2022.naacl-srw)

Challenge:	Song translation requires both translation of lyrics and alignment of music notes . human translators of songs need to have a mastery of cultural traditions and the poetic usage of both source and target languages .
Approach:	They propose a model that can model lyric translation and lyrics-melody alignment . they use an encoder-decoder framework that can translate lyrics and determine number of aligned notes .
Outcome:	The proposed framework can translate lyrics and determine the number of aligned notes at each decoding step.

Copied to clipboard

Mengsay Loem, Sho Takase, Masahiro Kaneko, Naoaki Okazaki

Advances and Challenges in Unsupervised Neural Machine Translation (2021.eacl-tutorials)

Challenge:	Recent studies indicated that neural methods are governed by the scaling law for the amount of training data.
Approach:	They propose a low-cost strategy to augment training data for abstractive summarization tasks by extracting summarized text and paraphrasing it.
Outcome:	The proposed strategy outperforms back-translation and self-training and is more cost-efficient when training data is small.

Copied to clipboard

Rui Wang, Hai Zhao

Machine Translation for Low-Resource Languages through Monolingual Data and LLM: A Case Study of English-to-Basque (2026.eacl-srw)

Challenge:	Unsupervised neural machine translation (UNMT) has achieved impressive results, but there are still several challenges for the technology.
Approach:	They present a framework for unsupervised neural machine translation (UNMT) they examine the latest progress and challenges of UNMT and examine how it holds up .
Outcome:	The proposed method has achieved impressive results but still faces challenges.

Copied to clipboard

Nam Luu, Aitor Soroa, German Rigau, Ondřej Bojar

NICT’s participation to WAT 2019: Multilingualism and Multi-step Fine-Tuning for Low Resource NMT (D19-52)

Challenge:	Existing LLMs do not translate well from English to Basque, but they yield an acceptable performance in the reverse direction.
Approach:	They propose to use a Basque monolingual corpora to train an LLM-based MT system . they use 'sovereignty fine tuning' to generate parallel corporata, and then use preference optimization .
Outcome:	The proposed system improves translation quality in English-to-Basque direction while requiring limited data for low-resource languages.

Copied to clipboard

Raj Dabre, Eiichiro Sumita

Simulated multiple reference training improves low-resource machine translation (2020.emnlp-main)

Challenge:	In this paper, we describe our submissions for the following tasks: English–Tamil translation and Russian–Japanese translation.
Approach:	They propose to use multilingual domain adaptation and back-translation to improve translations in Russian–Japanese and English–Tamil.
Outcome:	The proposed techniques perform better in Russian–Japanese and English–Tamil translation tasks.

Copied to clipboard

Huda Khayrallah, Brian Thompson, Matt Post, Philipp Koehn

KNU-HYUNDAI’s NMT system for Scientific Paper and Patent Tasks onWAT 2019 (D19-52)

Challenge:	Existing valid translations for a given sentence are limited by a single reference translation, causing data sparsity in low-resource settings.
Approach:	They propose a method that approximates the full space of possible translations by sampling a paraphrase of the reference sentence from a MT model and training it to predict the paraphraser’s distribution over possible tokens.
Outcome:	The proposed method improves in low-resource settings and is complementary to back-translation.

Copied to clipboard

Cheoneum Park, Young-Jun Jung, Kihoon Kim, Geonyeong Kim, Jae-Won Jeon, Seongmin Lee, Junseok Kim, Changki Lee

Efficient Semi-supervised Consistency Training for Natural Language Understanding (2022.naacl-industry)

Challenge:	We submitted our transformer-based neural machine translation system to the translation tasks of the 6th workshop on Asian Translation (WAT 2019).
Approach:	They propose a transformer-based neural machine translation system for Chinese-Japanese, English-Japanese, and Korean->Japanoise translation tasks.
Outcome:	The proposed system performed well on the two translation tasks and was ranked first in terms of the BLEU scores in all the JPC2 subtasks.

Copied to clipboard

George Leung, Joshua Tan

Data Augmentation Techniques for Machine Translation of Code-Switched Texts: A Comparative Study (2023.findings-emnlp)

Challenge:	Manually labeled training data is expensive, noisy, and often scarce . semi-supervised learning methods can be used to improve model performance .
Approach:	They explore different methods for consistency training on unlabeled data . they use human paraphrasing, back-translation, and dropout to augment unlabed data.
Outcome:	The proposed methods outperform purely supervised learning on unlabeled data.

Copied to clipboard

Injy Hamed, Nizar Habash, Thang Vu

Facebook AI’s WAT19 Myanmar-English Translation Task Submission (D19-52)

Challenge:	Code-switching (CSW) text generation is a popular solution to address data scarcity.
Approach:	They compare linguistic theories, lexical replacements and back-translation approaches to Egyptian Arabic-English CSW.
Outcome:	The proposed methods perform best on machine translation and quality evaluation.

Copied to clipboard

Peng-Jen Chen, Jiajun Shen, Matthew Le, Vishrav Chaudhary, Ahmed El-Kishky, Guillaume Wenzek, Myle Ott, Marc’Aurelio Ranzato

Back-Translation as Strategy to Tackle the Lack of Corpus in Natural Language Generation from Semantic Representations (D19-63)

Challenge:	Using back-translation, we can improve generalization by using noisy channel re-ranking and ensembling.
Approach:	They propose to use BPE-based transformer models to leverage monolingual data to improve generalization and use noisy channel re-ranking and ensembling to improve results.
Outcome:	The proposed system improves on the baseline system trained exclusively on the provided small parallel dataset, and the human evaluation and BLEU score are higher.

Copied to clipboard

Marco Antonio Sobrevilla Cabezudo, Simon Mille, Thiago Pardo

Comparison of Grammatical Error Correction Using Back-Translation Models (2021.naacl-srw)

Challenge:	Abstract Meaning Representation and Brazilian Portuguese (BP) are selected as semantic representation and language, respectively.
Approach:	They propose to use Brazilian Portuguese and Abstract Meaning Representation as semantic representations for NLG.
Outcome:	The proposed methods were evaluated on two datasets (one automatically generated and another human-generated) to compare the performance in a real context.

Copied to clipboard

Aomi Koyama, Kengo Hotate, Masahiro Kaneko, Mamoru Komachi

Sentence Concatenation Approach to Data Augmentation for Neural Machine Translation (2021.naacl-srw)

Challenge:	Currently, a mainstream approach to generate pseudo data is back-translation (BT).
Approach:	They propose to use back-translation to generate pseudo data that contains grammatical and ungrammatically produced sentences.
Outcome:	The proposed methods improve or interpolate the performance of each error type compared with a single BT model with different seeds.

Copied to clipboard

Seiichiro Kondo, Kengo Hotate, Tosho Hirasawa, Masahiro Kaneko, Mamoru Komachi

An Effective Approach to Unsupervised Machine Translation (P19-1)

Challenge:	Neural machine translation is known to show poor performance at long sentence translations . however, when the sentence length exceeds a certain value, the quality of NMT becomes inferior to that of statistical machine translation.
Approach:	They propose a method that uses given parallel corpora as train data to generate long sentences by concatenating two sentences at random.
Outcome:	The proposed method improves translation quality more when combined with back-translation.

Copied to clipboard

Mikel Artetxe, Gorka Labaka, Eneko Agirre

Using Neural Machine Translation Methods for Sign Language Translation (2022.acl-srw)

Challenge:	a recent research line has managed to train both unsupervised and unsupervised machine translation systems using monolingual corpora only.
Approach:	They propose to use monolingual corpora to train both unsupervised and unsupervised machine translation systems.
Outcome:	The proposed system achieves 22.5 BLEU points in English-to-German WMT 2014, 5.5 points more than the previous best unsupervised system, and 0.5 points more in the (supervised) shared task winner back in 2014.

Copied to clipboard

Galina Angelova, Eleftherios Avramidis, Sebastian Möller

An Exploration of Data Augmentation and Sampling Techniques for Domain-Agnostic Question Answering (D19-58)

Challenge:	Sign languages are the main medium of exchanging information for the deaf and hard of hearing.
Approach:	They propose to use two NMT architectures to train models on parallel German Sign Language corpora . they achieve substantial improvement in BLEU scores for the models trained on the two corporales .
Outcome:	The proposed models achieve significant improvements on the two corpora trained on the german sign language . the proposed models outperform the models trained on both corporales .

Copied to clipboard

Shayne Longpre, Yi Lu, Zhucheng Tu, Chris DuBois

Universal Neural Machine Translation for Extremely Low Resource Languages (N18-1)

Challenge:	XLNet model is domain-agnostic for the MRQA 2019 Shared Task . a negative sampling technique is particularly effective for datasets that include unanswerable questions .
Approach:	They develop a domain-agnostic question answering model for the MRQA 2019 Shared Task . they use large pre-trained language models, various data sampling strategies and query and context paraphrases generated by back-translation .
Outcome:	The proposed model achieves second best Exact Match and F1 in the MRQA leaderboard competition.

Copied to clipboard

Jiatao Gu, Hany Hassan, Jacob Devlin, Victor O.K. Li

On Synthetic Data for Back Translation (2022.naacl-main)

Challenge:	a novel multilingual approach to machine translation is proposed for low resource languages . the proposed approach can achieve 23 BLEU on Romanian-English WMT2016 using a tiny parallel corpus of 6k sentences compared to the 18 BLUE of strong baseline system .
Approach:	They propose a transfer-learning approach to share lexical and sentence representations across multiple source languages into one target language.
Outcome:	The proposed approach achieves 23 BLEU on Romanian-English WMT2016 using a tiny parallel corpus of 6k sentences compared to the 18 BLUE of strong baseline system which uses multi-lingual training and back-translation.

Copied to clipboard

Jiahao Xu, Yubin Ruan, Wei Bi, Guoping Huang, Shuming Shi, Lihui Chen, Lemao Liu

Back-Translation Sampling by Targeting Difficult Words in Neural Machine Translation (D18-1)

Challenge:	Existing studies on back translation (BT) focus on beam search or random sampling . a new method to generate synthetic data with a backward model is proposed to improve BT performance.
Approach:	They propose a method to generate synthetic data to trade off quality and importance factors . back translation (BT) is one of the most significant technologies in NMT research fields .
Outcome:	The proposed method outperforms the baseline methods on WMT14 DE-EN, EN-DE, and RU-EN benchmark tasks.

Copied to clipboard

Marzieh Fadaee, Christof Monz

ChrEn: Cherokee-English Machine Translation for Endangered Language Revitalization (2020.emnlp-main)

Challenge:	Neural machine translation (NMT) uses a sequence-to-sequence model to generate synthetic data.
Approach:	They propose a method that adds synthetic data to sentences with high prediction loss during training and a variety of sampling strategies targeting difficult-to-predict words.
Outcome:	The proposed method improves translation quality by up to 1.7 and 1.2 Bleu points over back-translation using random sampling for German-English and English-German, respectively.

Copied to clipboard

Shiyue Zhang, Benjamin Frey, Mohit Bansal

Understanding Back-Translation at Scale (D18-1)

Challenge:	Cherokee is a highly endangered Native American language spoken by the Cherokee people . there are only 2,000 fluent first language Cherokee speakers remaining in the world .
Approach:	They propose a Cherokee-English parallel dataset to facilitate machine translation between Cherokee and English.
Outcome:	The proposed dataset compares Cherokee-English and English-Cherokee machine translation systems . the results show that the datasets are low-resource and low-cost compared to other datasets .

Copied to clipboard

Sergey Edunov, Myle Ott, Michael Auli, David Grangier

Vector Space Interpolation for Query Expansion (2022.aacl-short)

Challenge:	An effective method to improve neural machine translation with monolingual data is to augment the parallel training corpus with back-translations of target language sentences.
Approach:	They propose to augment parallel training corpus with back-translations of target language sentences to improve neural machine translation with monolingual data.
Outcome:	The proposed method achieves a state-of-the-art of 35 BLEU on the WMT’14 English-German test set.

Copied to clipboard

Deepanway Ghosal, Somak Aditya, Sandipan Dandapat, Monojit Choudhury

Representation Learning

Challenge:	Topic-sensitive query set expansion is crucial for queries related to sensitive and emerging topics.
Approach:	They propose a method for topic-sensitive query set expansion using vector space interpolation.
Outcome:	The proposed method generates new queries about the sensitive topic by incorporating set diversity, which is not captured by traditional sentence-level augmentation methods such as paraphrasing or back-translation.

A Large-Scale Benchmark for Vietnamese Sentence Paraphrases (2025.findings-naacl)

Copied to clipboard

Sang Quang Nguyen, Kiet Van Nguyen

Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Data (2021.acl-long)

Challenge:	1.2M original–paraphrase pairs were generated using a hybrid approach to generate high-quality paraphrases.
Approach:	They present a high-quality Vietnamese dataset for sentence paraphrasing . they used automatic paraphrase generation and manual evaluation to ensure high quality .
Outcome:	The proposed dataset is the first large-scale study on Vietnamese paraphrasing . it combines automatic paraphrase generation with manual evaluation to ensure high quality .

Copied to clipboard

Wei-Jen Ko, Ahmed El-Kishky, Adithya Renduchintala, Vishrav Chaudhary, Naman Goyal, Francisco Guzmán, Pascale Fung, Philipp Koehn, Mona Diab

Syn-QG: Syntactic and Shallow Semantic Rules for Question Generation (2020.acl-main)

Challenge:	linguistic overlap between low-resource languages and high-resourced languages is a major obstacle for training high-quality machine translation systems.
Approach:	They exploit linguistic overlap to facilitate translation to and from low-resource languages . they use monolingual data and parallel data in related high-resourced languages based on their method .
Outcome:	The proposed method significantly improves translation into low-resource language compared to baselines on 7 languages from three different language families.

Copied to clipboard

Kaustubh Dhole, Christopher D. Manning

Improving Back-Translation with Uncertainty-based Confidence Estimation (D19-1)

Challenge:	Question Generation is a simple syntactic transformation but many aspects of semantics influence what questions are good to form.
Approach:	They propose a set of syntactic rules which transform declarative sentences into question-answer pairs.
Outcome:	The proposed system generates a larger number of highly grammatical and relevant questions than existing QG systems.

Copied to clipboard

Shuo Wang, Yang Liu, Chao Wang, Huanbo Luan, Maosong Sun

Robust Explanations for User Trust in Enterprise NLP Systems (2026.acl-industry)

Challenge:	Despite the success of low-resource neural machine translation, there is a data scarcity problem in many languages . large-scale, high-quality, and widecoverage bilingual corpora do not exist for most language pairs .
Approach:	They propose to quantify confidence of NMT models based on model uncertainty . they propose to use uncertainty-based confidence measures to improve back-translation .
Outcome:	The proposed model outperforms conventional statistical machine translation (SMT) on Chinese-English and English-German translation tasks.

Copied to clipboard

Guilin Zhang, Kai Zhao, Jeffrey Friedman, Xu Chu, Amine Anoun

Generative Data Augmentation for Commonsense Reasoning (2020.findings-emnlp)

Challenge:	Existing studies on explanation stability under real user noise are limited . decoder LLMs produce significantly more stable explanations than encoder baselines .
Approach:	They propose a black-box robustness evaluation framework for token-level explanations based on leave-one-out occlusion . they propose to operationalize explanation robustness with top-token flip rate under realistic perturbations at multiple severity levels .
Outcome:	The proposed framework is compared with baseline models and encoder and decoder families.

Copied to clipboard

Yiben Yang, Chaitanya Malaviya, Jared Fernandez, Swabha Swayamdipta, Ronan Le Bras, Ji-Ping Wang, Chandra Bhagavatula, Yejin Choi, Doug Downey

Improving Unsupervised Word-by-Word Translation with Language Model and Denoising Autoencoder (D18-1)

Challenge:	Recent advances in commonsense reasoning depend on large-scale human-authored training data.
Approach:	They propose a generative data augmentation technique that augments human-authored training data by using pretrained language models.
Outcome:	The proposed technique outperforms existing methods on commonsense reasoning benchmarks and enhances out-of-distribution generalization.

Copied to clipboard

Yunsu Kim, Jiahui Geng, Hermann Ney

Summarize and Generate to Back-translate: Unsupervised Translation of Programming Languages (2023.eacl-main)

Challenge:	Unsupervised learning of cross-lingual word embeddings has fundamental limitations in translating sentences.
Approach:	They propose a method to improve word-by-word translation of cross-lingual embeddings using monolingual corpora without any back-translation.
Outcome:	The proposed system surpasses state-of-the-art unsupervised translation systems without costly iterative training.

Copied to clipboard

Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, Kai-Wei Chang

Unsupervised Bilingual Word Embedding Agreement for Unsupervised Neural Machine Translation (P19-1)

Challenge:	Recent developments of multilingual pre-trained sequence-to-sequence models for programming languages have been effective for a broad spectrum of downstream software engineering tasks.
Approach:	They propose to combine a source-to-target model with a target-tosource model trained in parallel.
Outcome:	The proposed approach performs competitively with state-of-the-art methods.

Copied to clipboard

Haipeng Sun, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita, Tiejun Zhao

Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabularies (P19-1)

Challenge:	Unsupervised bilingual word embedding (UBWE) has helped unsupervised neural machine translation (UNMT) achieve remarkable results in several language pairs.
Approach:	They propose two methods that train UNMT with UBWE agreement . they propose to use UBwe to initialize word embedding in UNMT .
Outcome:	The proposed methods outperform conventional methods on several language pairs.

Copied to clipboard

Yunsu Kim, Yingbo Gao, Hermann Ney

Improved Zero-shot Neural Machine Translation via Ignoring Spurious Correlations (P19-1)

Challenge:	Existing approaches to transfer a pretrained NMT model to a new, unrelated language without shared vocabularies are limited to cognate languages.
Approach:	They propose to transfer a pretrained NMT model to a new, unrelated language without shared vocabularies by using cross-lingual word embedding and injecting artificial noises.
Outcome:	The proposed methods outperform multilingual joint training by a large margin in five low-resource translation tasks.

Copied to clipboard

Jiatao Gu, Yong Wang, Kyunghyun Cho, Victor O.K. Li

GFST: Gender-Filtered Self-Training for More Accurate Gender in Translation (2021.emnlp-main)

Challenge:	Existing approaches to train a multilingual NMT model for low-resource languages are lacking in terms of number of supervised examples.
Approach:	They propose to use decoder pre-training and back-translation to solve the degeneracy problem by analyzing spurious correlations between source and decoded sentences.
Outcome:	The proposed methods show significant improvement over the pivot-based approach on three challenging multilingual datasets.

Copied to clipboard

Prafulla Kumar Choubey, Anna Currey, Prashant Mathur, Georgiana Dinu

Putting words into the system’s mouth: A targeted attack on neural machine translation using monolingual data poisoning (2021.findings-acl)

Challenge:	Recent studies have focused on gender bias in neural machine translation (NMT) incorrectly gendered translations can reflect or amplify social biases.
Approach:	They propose to use a monolingual corpus to generate gender-specific pseudo-parallel corpora and filter them to improve gender translation accuracy.
Outcome:	The proposed approach improves gender accuracy without damaging generic quality on translations from English into five languages.

Copied to clipboard

Jun Wang, Chang Xu, Francisco Guzmán, Ahmed El-Kishky, Yuqing Tang, Benjamin Rubinstein, Trevor Cohn

Robustness in NLP

Challenge:	Neural machine translation systems are known to be vulnerable to adversarial test inputs, however, they are also vulnerable to training attacks.
Approach:	They propose a poisoning attack in which a malicious adversary inserts a small poisoned sample of monolingual text into a training set of a system trained using back-translation.
Outcome:	The proposed attack is based on two methods that can be used to craft poisoned examples.

The Source-Target Domain Mismatch Problem in Machine Translation (2021.eacl-main)

Copied to clipboard

Jiajun Shen, Peng-Jen Chen, Matthew Le, Junxian He, Jiatao Gu, Myle Ott, Michael Auli, Marc’Aurelio Ranzato

N-Shot Learning for Augmenting Task-Oriented Dialogue State Tracking (2022.findings-acl)

Challenge:	Despite the interconnected world we live in, people in different places talk about different things in different parts of the world.
Approach:	They propose a metric to quantify the effect of local context in machine translation and propose measurable results.
Outcome:	The proposed metric can be used to quantify the effect of local context on the use of language in machine translation systems on low resource languages.

Copied to clipboard

Ibrahim Aksu, Zhengyuan Liu, Min-Yen Kan, Nancy Chen

Dialogue Systems & Conversational Agents

Challenge:	augmentation of task-oriented dialogues has followed standard methods for plain-text despite its richly annotated structure.
Approach:	They propose an augmentation framework that utilizes belief state annotations to match turns from various dialogues and form new synthetic dialogues in a bottom-up manner.
Outcome:	The proposed framework performs better on seen values and more robust to unseen values on n-shot training scenarios.

A Reinforcement Learning Approach to Improve Low-Resource Machine Translation Leveraging Domain Monolingual Data (2024.lrec-main)

Copied to clipboard

Hongxiao Zhang, Mingtong Liu, Chunyou Li, Yufeng Chen, Jinan Xu, Ming Zhou

Pushing Paraphrase Away from Original Sentence: A Multi-Round Paraphrase Generation Approach (2021.findings-acl)

Challenge:	Existing methods for fine-tuning domain adaptation have overfitting problem in low-resource domains . lack of parallel data makes it difficult for model to learn domain-specific knowledge .
Approach:	They propose a Reinforcement Learning Domain Adaptation method for Neural Machine Translation that uses in-domain source monolingual data to make up for the lack of parallel data.
Outcome:	The proposed method can alleviate overfitting and reinforce the model to learn domain-specific knowledge.

Copied to clipboard

Zhe Lin, Xiaojun Wan

Handling Syntactic Divergence in Low-resource Machine Translation (D19-1)

Challenge:	Recent years, neural paraphrase generation models have demonstrated superior performance, but the output paraphrase still lacks diversity.
Approach:	They propose a back-translation guided multi-round paraphrase generation framework which leverages multi- round paraphrases to improve diversity while preserving semantic information.
Outcome:	The proposed model improves diversity while preserving semantic information.

Copied to clipboard

Chunting Zhou, Xuezhe Ma, Junjie Hu, Graham Neubig

Unveiling the Power of Source: Source-based Minimum Bayes Risk Decoding for Neural Machine Translation (2025.acl-long)

Challenge:	Existing approaches to neural machine translation (NMT) are dependent on limited parallel data, and can be difficult to use for many language pairs.
Approach:	They propose a method where target-language sentences are re-ordered to match the order of the source and used as an additional source of training-time supervision.
Outcome:	The proposed method improves on simulated low-resource Japanese-to-English and real low-demand Uyghur-to English scenarios.

Copied to clipboard

Boxuan Lyu, Hidetaka Kamigaito, Kotaro Funakoshi, Manabu Okumura

Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translation (2021.naacl-main)

Challenge:	Maximum a posteriori decoding aims to maximize the estimated posterior probability, but high estimated probability does not always lead to high translation quality.
Approach:	They propose a method that seeks hypotheses with the highest expected utility by using quasi-sources as “support hypothese . they propose sMBR decoding which utilizes a reference-free quality estimation metric as the utility function.
Outcome:	The proposed approach outperforms QE reranking and the standard MBR decoding.

Copied to clipboard

Hirofumi Inaguma, Tatsuya Kawahara, Shinji Watanabe

A Comparison between Pre-training and Large-scale Back-translation for Neural Machine Translation (2021.findings-acl)

Challenge:	End-to-end speech translation models can be trained to leverage source text . however, since the input modalities are different, it is difficult to leverage the source text successfully.
Approach:	They propose to leverage source transcriptions via pre-training and joint training with ASR and NMT tasks.
Outcome:	The proposed model predicts paraphrased transcriptions as an auxiliary task with a single decoder.

Copied to clipboard

Dandan Huang, Kun Wang, Yue Zhang

Data Augmentation for Voice-Assistant NLU using BERT-based Interchangeable Rephrase (2021.eacl-main)

Challenge:	BERT is a promising technique to improve NMT, but how it outperforms standard NMT is understudied.
Approach:	We compare MT engines trained with pre-trained BERT and back-translation with incrementally larger amounts of data.
Outcome:	The proposed technique outperforms standard NMT models on morphology and syntax.

Copied to clipboard

Akhila Yerukola, Mason Bretan, Hongxia Jin

Speech & Audio in NLP

Challenge:	a data augmentation technique is used to boost performance on spoken language understanding tasks.
Approach:	They propose a data augmentation technique based on byte pair encoding and a BERT-like self-attention model to boost performance on spoken language understanding tasks.
Outcome:	The proposed method performs well on domain and intent classification tasks for a voice assistant and in a user-study focused on utterance naturalness and semantic similarity.

Synonym relations affect object detection learned on vision-language data (2024.findings-naacl)

Copied to clipboard

Giacomo Nebbia, Adriana Kovashka

Visual Data in NLP

Challenge:	a recent study shows that vision-language models that accept textual input are not robust to variations in how input is provided.
Approach:	They propose two approaches to improve vision-language object detectors' performance . they use back-translation and class embedding enrichment to improve their models .
Outcome:	The proposed approaches improve performance on synonyms from mAP@0.3=33.87% to 37.93%.

Two Parents, One Child: Dual Transfer for Low-Resource Neural Machine Translation (2021.findings-acl)

Copied to clipboard

Meng Zhang, Liangyou Li, Qun Liu

On the Complementarity between Pre-Training and Back-Translation for Neural Machine Translation (2021.findings-emnlp)

Challenge:	Neural machine translation suffers when parallel data is scarce for training . a new framework to transfer multiple sources of auxiliary data is proposed .
Approach:	They propose a framework to transfer multiple sources of auxiliary data from high-resource parallel data to low-resourced translation models using pretrained language models.
Outcome:	The proposed approach yields consistent improvements over strong competitors for multiple translation directions.

Copied to clipboard

Xuebo Liu, Longyue Wang, Derek F. Wong, Liang Ding, Lidia S. Chao, Shuming Shi, Zhaopeng Tu

Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation (2020.acl-main)

Challenge:	Experimental results show that PT and BT are nicely complementary to each other.
Approach:	They introduce two probing tasks for PT and BT respectively and investigate their complementarity.
Outcome:	The proposed methods establish state-of-the-art on the WMT16 English-Romanian and English-Russian benchmarks.

Copied to clipboard

Aditya Siddhant, Ankur Bapna, Yuan Cao, Orhan Firat, Mia Chen, Sneha Kudugunta, Naveen Arivazhagan, Yonghui Wu

On The Evaluation of Machine Translation Systems Trained With Back-Translation (2020.acl-main)

Challenge:	Existing multilingual NMT approaches do not utilize the abundance of monolingual data, especially in low-resource languages.
Approach:	They propose to combine monolingual data with self-supervision to pre-train translation models and fine-tune on small amounts of supervised data.
Outcome:	The proposed approach improves translation quality of low-resource languages and zero-shot translation quality.

Copied to clipboard

Sergey Edunov, Myle Ott, Marc’Aurelio Ranzato, Michael Auli

AugVic: Exploiting BiText Vicinity for Low-Resource NMT (2021.findings-acl)

Challenge:	Back-translation is a data augmentation technique that can be used to improve neural machine translation systems.
Approach:	They propose to combine back-translation with a language model score to measure fluency.
Outcome:	The proposed method improves translation quality of natural text and translationese according to professional translators.

Copied to clipboard

Tasnim Mohiuddin, M Saiful Bari, Shafiq Joty

Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout (N19-1)

Challenge:	Neural Machine Translation (NMT) systems often exhibit poor performance due to the lack of large bitext training corpora in low-resource languages.
Approach:	They propose a data augmentation framework which exploits the vicinal samples of the given bitext without using extra monolingual data explicitly.
Outcome:	The proposed framework can diversify in-domain bitext data with finer level control on four low-resource language pairs.

Copied to clipboard

Hao Tan, Licheng Yu, Mohit Bansal

Bi-SimCut: A Simple Strategy for Boosting Neural Machine Translation (2022.naacl-main)

Challenge:	Existing approaches perform significantly worse in unseen environments compared to seen ones.
Approach:	They propose to use a ‘environmental dropout’ method to generate unseen triplets to generate new paths and instructions to generalize the agent.
Outcome:	The proposed agent outperforms the state-of-the-art approaches on the private unseen test set and is ranked top on the leaderboard.

Copied to clipboard

Pengzhi Gao, Zhongjun He, Hua Wu, Haifeng Wang

Adaptation of Back-translation to Automatic Post-Editing for Synthetic Data Generation (2021.eacl-main)

Challenge:	BLEU scores of 31.16 for ende and 38.37 for deen on the IWSLT14 dataset, 30.78 for entde, 35.15 for de en and 27.17 for zhen .
Approach:	They propose a bidirectional pretraining and unidirectional finetuning procedure to boost NMT performance.
Outcome:	The proposed method achieves strong translation performance across five datasets.

Copied to clipboard

WonKee Lee, Baikjin Jung, Jaehun Shin, Jong-Hyeok Lee

Non-Parametric Unsupervised Domain Adaptation for Neural Machine Translation (2021.findings-emnlp)

Challenge:	Automated Post-Editing (APE) aims to correct errors in the output of a given machine translation system.
Approach:	They propose two new methods of synthesizing additional MT outputs by adapting back-translation to the APE task, obtaining robust enlargements of existing synthetic APE training dataset.
Outcome:	The proposed methods improve translation quality on the English-German APE task by enlarging the existing training dataset.

Copied to clipboard

Xin Zheng, Zhirui Zhang, Shujian Huang, Boxing Chen, Jun Xie, Weihua Luo, Jiajun Chen

Filtering Back-Translated Data in Unsupervised Neural Machine Translation (2020.coling-main)

Challenge:	kNN-MT is a non-parametric method that uses nearest neighbor retrieval to translate out-of-domain sentences, rare words, etc.
Approach:	They propose a framework that directly uses in-domain monolingual sentences to build an effective datastore for k-nearest-neighbor retrieval.
Outcome:	The proposed framework improves translation accuracy with target-side monolingual data while achieving comparable performance with back-translation.

Copied to clipboard

Jyotsana Khatri, Pushpak Bhattacharyya

Unsupervised Extraction of Partial Translations for Neural Machine Translation (N19-1)

Challenge:	Current state of the art approaches for unsupervised neural machine translation (NMT) use only monolingual data for training.
Approach:	They propose an approach to filter back-translated data as part of the training process of unsupervised neural machine translation (NMT) they propose a weight component based on the quality of pseudo parallel sentence pairs generated in back-translation phase.
Outcome:	The proposed approach improves the training performance of unsupervised neural machine translation systems by giving weight to good pseudo parallel sentence pairs in the back-translation phase.

Copied to clipboard

Benjamin Marie, Atsushi Fujita

TreeMix: Compositional Constituency-based Data Augmentation for Natural Language Understanding (2022.naacl-main)

Challenge:	Neural machine translation systems usually require a large quantity of bilingual parallel data for training.
Approach:	They propose an algorithm for extracting from monolingual data what they call partial translations . partial translation is a pair of source and target sentences that contain sequences of tokens that are translations of each other.
Outcome:	The proposed algorithm extracts from monolingual data what we call partial translations . it takes only source and target monolingual datasets as input .

Copied to clipboard

Le Zhang, Zichao Yang, Diyi Yang

Low-Resource NLP

Challenge:	Existing data augmentation methods miss the important characteristic of compositionality, meaning of a complex expression is built from its sub-parts.
Approach:	They propose a compositional data augmentation approach for natural language understanding called TreeMix that leverages constituency parsing tree to decompose sentences into constituent sub-structures and the Mixup data enhancing technique to recombine them to generate new sentences.
Outcome:	The proposed approach outperforms current state-of-the-art methods on text classification and SCAN.

Unifying Input and Output Smoothing in Neural Machine Translation (2020.coling-main)

Copied to clipboard

Yingbo Gao, Baohao Liao, Hermann Ney

Benchmark Creation for Aspect-Based Sentiment Analysis in Low-Resource Odia Language and Evaluation through Fine-Tuning of Multilingual Models (2025.coling-main)

Challenge:	Recent methods that smooth input and output of neural machine translation systems bring significant improvements in performance.
Approach:	They propose a method that replaces one-hot representations with soft posterior distributions of an external language model, smoothing the input of machine translation systems.
Outcome:	The proposed method improves translation performance on small datasets and larger datasets.

Copied to clipboard

Lipika Dewangan, Zoyah Afsheen Sayeed, Chandresh Maurya

Sentiment Analysis

Challenge:	Aspect-based sentiment analysis is underexplored in low-resource languages such as Odia . a dataset is annotated for two tasks: Aspect Term Extraction (ATE) and Aspect Polarity Classification (APC)
Approach:	They propose to use a dataset for aspect-based sentiment analysis in Odia . they use ensemble data augmentation and a fine-tuned paraphrase generation model .
Outcome:	The proposed dataset is annotated for two tasks: ATE and APC . the proposed dataset will spur more work for the ABSA task in Odia .

A Corpus for Automatic Readability Assessment and Text Simplification of German (2020.lrec-1)

Copied to clipboard

Alessia Battisti, Dominik Pfütze, Andreas Säuberli, Marek Kostrzewa, Sarah Ebling

Unsupervised Data Augmentation with Naive Augmentation and without Unlabeled Data (2021.emnlp-main)

Challenge:	Using monolingual-only data, we can automate readability assessment and text simplification of simplified language.
Approach:	They present a corpus for automatic readability assessment and automatic text simplification for German using parallel and monolingual data.
Outcome:	The proposed corpus is compiled from web sources and contains information on text structure, typography, font style, and images.

Copied to clipboard

David Lowell, Brian Howard, Zachary C. Lipton, Byron Wallace

Low-Resource NLP

Challenge:	Unsupervised Data Augmentation (UDA) is a semisupervised learning method that penalizes differences between a model's predictions on unlabeled examples and corresponding 'noised' examples produced via data augmentation.
Approach:	They propose to use a consistency loss to penalize differences between models' predictions on unlabeled and unlabed examples to enforce consistency between models and their perturbed counterparts.
Outcome:	The proposed method is able to penalize differences between models' outputs on unlabeled and unlabed examples without complex data augmentation.

On the Language Coverage Bias for Neural Machine Translation (2021.findings-acl)

Copied to clipboard

Shuo Wang, Zhaopeng Tu, Zhixing Tan, Shuming Shi, Maosong Sun, Yang Liu

Improving Language Model Integration for Neural Machine Translation (2023.findings-acl)

Challenge:	Language coverage bias is important for neural machine translation because of the target-original training data.
Approach:	They propose two approaches to alleviate the language coverage bias problem by explicitly distinguishing between the source-and target-original training data.
Outcome:	The proposed methods improve translation tasks on both back-and forward-translation and their tagged variants.

Copied to clipboard

Christian Herold, Yingbo Gao, Mohammad Zeineldeen, Hermann Ney

Language Models

Challenge:	Existing methods to integrate external language models into machine translation systems have been based on the assumption that the external model learns an implicit target-side language model at decoding time.
Approach:	They transfer this concept to the task of machine translation and compare it with the most prominent way of including additional monolingual data - namely back-translation.
Outcome:	The proposed approach outperforms the most prominent way of including additional monolingual data, namely back-translation.

AraBench: Benchmarking Dialectal Arabic-English Machine Translation (2020.coling-main)

Copied to clipboard

Hassan Sajjad, Ahmed Abdelali, Nadir Durrani, Fahim Dalvi

Iterative Constrained Back-Translation for Unsupervised Domain Adaptation of Machine Translation (2022.coling-1)

Challenge:	Existing efforts to translate Arabic dialects to English are limited due to the lack of evaluation benchmarks.
Approach:	They propose an evaluation suite for Arabic to English machine translation using existing Arabic resources.
Outcome:	The evaluation suite for Arabic to English machine translation is based on existing evaluation benchmarks.

Copied to clipboard

Hongxiao Zhang, Hui Huang, Jiale Gao, Yufeng Chen, Jinan Xu, Jian Liu

Iterative Domain-Repaired Back-Translation (2020.emnlp-main)

Challenge:	Existing back-translation methods focus on in-domain lexical knowledge, which may lead to poor translation of unseen in- domain words.
Approach:	They propose an iterative constrained back-translation method to incorporate in-domain lexical knowledge into synthetic parallel data from BT.
Outcome:	The proposed method improves the BLEU score by up to 3.08 on four domains.

Copied to clipboard

Hao-Ran Wei, Zhirui Zhang, Boxing Chen, Weihua Luo

Dynamic Data Selection and Weighting for Iterative Back-Translation (2020.emnlp-main)

Challenge:	Existing studies show that NMT models perform poorly in specific domains when in-domain parallel corpora are scarce or nonexistent.
Approach:	They propose an iterative domain-repaired back-translation framework to refine translations in bilingual data by round-trip translating monolingual sentences.
Outcome:	The proposed framework achieves 15.79 and 4.47 BLEU improvements over unadapted models and back-translation in domain-specific translations.

Copied to clipboard

Zi-Yi Dou, Antonios Anastasopoulos, Graham Neubig

Simplifying Translations for Children: Iterative Simplification Considering Age of Acquisition with LLMs (2024.findings-acl)

Challenge:	Experimental results demonstrate that our methods achieve improvements of up to 1.8 BLEU points over competitive baselines.
Approach:	They propose a data selection and weighting strategy to iterate back-translation models and apply it to it . they use a target language to back-transcribe monolingual data, which is of high quality and reflect the target domain.
Outcome:	The proposed approach achieves 1.8 BLEU points over baselines on domain adaptation, low-resource, and high-resourced MT settings and on two language pairs.

Copied to clipboard

Masashi Oshika, Makoto Morishita, Tsutomu Hirao, Ryohei Sasano, Koichi Takeda

An Extensive Exploration of Back-Translation in 60 Languages (2023.findings-acl)

Challenge:	Neural machine translation (NMT) systems do not take into account the complexity of the words used to compose the translations.
Approach:	They propose a method that replaces high Age of Acquisitions words in translations with simpler words to match the user’s level.
Outcome:	The proposed method replaces high-AoA words with lower-Aa words while maintaining high BLEU and COMET scores.

Copied to clipboard

Paul McNamee, Kevin Duh

Evaluating Pre-training Objectives for Low-Resource Translation into Morphologically Rich Languages (2022.lrec-1)

Challenge:	Back-translation has been shown to improve model quality through the creation of synthetic training bitext.
Approach:	They use back-translation to train models from 60 languages into English . early studies showed promise of the technique and follow on studies have produced refinements .
Outcome:	a new study shows that back-translation improves translation quality in low-resource languages . the results are consistent with previous studies, though there are limitations .

Copied to clipboard

Prajit Dhar, Arianna Bisazza, Gertjan van Noord

Multilingual Unsupervised Neural Machine Translation with Denoising Adapters (2021.emnlp-main)

Challenge:	a lack of parallel data is a major limitation for Neural Machine Translation systems, especially for morphologically rich languages.
Approach:	They propose to leverage target monolingual data to overcome the lack of parallel data . they introduce a new technique called PT-Inflect to train NMT systems .
Outcome:	The proposed techniques outperform NMT systems trained on parallel data on four typologically diverse target languages.

Copied to clipboard

Ahmet Üstün, Alexandre Berard, Laurent Besacier, Matthias Gallé

Enhancing Taiwanese Hokkien Dual Translation by Exploring and Standardizing of Four Writing Systems (2024.lrec-main)

Challenge:	Multilingual unsupervised machine translation is a computationally expensive and hard to tune approach . auxiliary parallel data is used to train translation systems from monolingual data .
Approach:	They propose to use auxiliary parallel language pairs to train unsupervised machine translations . they propose to add auxiliary languages to pre-trained mBART-50 models with denoising adapters .
Outcome:	The proposed approach is on-par with back-translation and allows adding unseen languages incrementally.

Copied to clipboard

Bo-Han Lu, Yi-Hsuan Lin, Annie Lee, Richard Tzong-Han Tsai

Visual Information Guided Zero-Shot Paraphrase Generation (2022.coling-1)

Challenge:	Currently, machine translation systems cater to high-resource languages (HRLs), while low-resourced languages (LRLs) like Taiwanese Hokkien are relatively under-explored.
Approach:	They propose to use a pre-trained LLaMA 2-7B model specialized in Traditional Mandarin Chinese to leverage orthographic similarities between Taiwanese Hokkien Han and Traditional Mandarin China.
Outcome:	The proposed model bridges the gap between Taiwanese Hokkien and other low-resource languages by using a pre-trained LLaMA 2-7B model and a monolingual corpus.

Copied to clipboard

Zhe Lin, Xiaojun Wan

Quick Back-Translation for Unsupervised Machine Translation (2023.findings-emnlp)

Challenge:	Several studies use different information as ”pivot” such as language, semantic representation and so on.
Approach:	They propose to use visual information as the "pivot" of back-translation to generate paraphrases using paired image-caption data.
Outcome:	The proposed model generates paraphrase with good relevancy, fluency and diversity . it is based on paired image-caption data and can train a paraphrasing model .

Copied to clipboard

Benjamin Brimacombe, Jiawei Zhou

Better OOV Translation with Bilingual Terminology Mining (P19-1)

Challenge:	Unsupervised machine translation models are limited by the run-time of autoregressive inference during back-translation and lack of synthetic data efficiency.
Approach:	They propose a two-for-one improvement to Transformer back-translation: Quick Back-Translation (QBT). QBT re-purposes the encoder as a generative model, and uses encoder-generated sequences to train the decoder.
Outcome:	Experiments on various WMT benchmarks show that QBT dramatically outperforms standard back-translation only method in terms of training efficiency for comparable translation qualities.

Copied to clipboard

Matthias Huck, Viktor Hangya, Alexander Fraser

Unsupervised Data Augmentation for Aspect Based Sentiment Analysis (2022.coling-1)

Challenge:	Unseen words, also called out-of-vocabulary words, are difficult for machine translation . byte-pair encoding can be used to represent OOVs, but they are often incorrectly translated .
Approach:	They propose to use monolingual data to improve the translation of unseen words . they use five target language words to mine target-language sentences .
Outcome:	The proposed system can be used to improve translation of out-of-vocabulary words (OOVs) the proposed system is trained on Europarl and can be fine-tuned to improve the translation quality.

Copied to clipboard

David Z. Chen, Adam Faulkner, Sahil Badyal

Information Extraction & Text Mining

Challenge:	Recent approaches to Aspect-based Sentiment Analysis (ABSA) perform the subtasks of aspect term extraction (ATE) and aspect sentiment classification (ASC) simultaneously.
Approach:	They introduce an adaptation of Unsupervised Data Augmentation in semi-supervised learning that performs both aspects of Aspect-based Sentiment Analysis (ABSA) and aspect sentiment classification (ASC) they show that simple augmentations applied to modest-sized datasets along with consistency training lead to competitive performance with current ABSA state-of-the-art in restaurant and laptop domains .
Outcome:	The proposed approach performs well on a span-level classification task with minimal training data.

An Inflectional Database for Gitksan (2022.lrec-1)

Copied to clipboard

Bruce Oliver, Clarissa Forbes, Changbing Yang, Farhan Samir, Edith Coates, Garrett Nicolai, Miikka Silfverberg

Exploiting Target Language Data for Neural Machine Translation Beyond Back Translation (2024.findings-acl)

Challenge:	In this paper, we build a database of partial inflection tables for Gitksan, a low-resource Indigenous language of Canada.
Approach:	They use Gitksan data in interlinear glossed format to build a database of partial inflection tables and enrich it with neural transformer reinflection models.
Outcome:	The proposed model improves the performance of the experimental data hallucination and back-translation techniques.

Copied to clipboard

Abudurexiti Reheman, Yingfeng Luo, Junhao Ruan, Chunliang Zhang, Anxiang Ma, Tong Xiao, JingBo Zhu

Improving Unsupervised Neural Machine Translation via Training Data Self-Correction (2024.lrec-main)

Challenge:	Neural Machine Translation (NMT) encounters challenges when translating in new domains and low-resource languages.
Approach:	They propose a variant of k-nearest neighbor machine translation that utilizes target language data by constructing a pseudo datastore.
Outcome:	The proposed method exhibits strong domain adaptation capability in both high-resource and low-resourced machine translation.

Copied to clipboard

Jinliang Lu, Jiajun Zhang

Tool learning via Inference-time Scaling and Cycle Verifier (2025.findings-acl)

Challenge:	Unsupervised neural machine translation models can generate mistakes during training . however, the quality of pseudo-parallel sentences cannot be guaranteed .
Approach:	They propose a method to improve the quality of pseudo-parallel sentences . they use token-level translations to correct mis-translated tokens .
Outcome:	Empirical results show that the proposed method outperforms baselines on widely used datasets.

Copied to clipboard

Xiaobo Liang, Wenjin Xie, Juntao Li, Wanfu Wang, Yibin Chen, Kehai Chen, Min Zhang

Challenge:	In inference-time scaling, Chain-of-Thought (CoT) data is scarce or even unavailable.
Approach:	They propose a method which establishes an inference cycle to synthesize user queries and CoT data.
Outcome:	The proposed method achieves a 75.4% pass rate and a 79.6% win rate using small models in StableToolBench.

AIR: Complex Instruction Generation via Automatic Iterative Refinement (2025.emnlp-main)

Copied to clipboard

Wei Liu, Yancheng He, Yu Li, Hui Huang, Chengwei Hu, Jiaheng Liu, Shilong Li, Wenbo Su, Bo Zheng