Papers by Eiichiro Sumita

Sentence-Level Agreement for Neural Machine Translation (P19-1)

Challenge:	Neural syntactic distance (NSD) is used to represent constituent trees using a sequence whose length is identical to the number of words in the sentence.
Approach:	They propose five strategies to improve NMT with explicit use of syntactic information . et al., 2014) propose a set of five strategies that incorporate syntastic information into the encoder and/or decoder of the baseline model.
Outcome:	The proposed strategies improve translation performance of the baseline model (+2.1 (En–Ja), +1.3 (Ja–En), +1.2 (En-Ch), and +1.0 (Ch–En) BLEU.

Copied to clipboard

Mingming Yang, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita, Min Zhang, Tiejun Zhao

Smoothing Dialogue States for Open Conversational Machine Reading (2021.emnlp-main)

Challenge:	Empirical results show that a sentence-level agreement module can significantly improve the performance of neural machine translation (NMT)
Approach:	They propose a sentence-level agreement module to minimize the difference between the representation of source and target sentences.
Outcome:	Empirical results show the proposed agreement module significantly improves translation performance.

Copied to clipboard

Zhuosheng Zhang, Siru Ouyang, Hai Zhao, Masao Utiyama, Eiichiro Sumita

Natural Language Interfaces

Challenge:	Existing studies train independent or pipeline systems for the two subtasks but are trivial by using hard-label decisions to activate question generation.
Approach:	They propose a method to smooth two dialogue states in one decoder and bridge decision making and question generation to provide a richer dialogue state reference.
Outcome:	The proposed method achieves state-of-the-art on the OR-ShARC dataset.

MY-AKKHARA: A Romanization-based Burmese (Myanmar) Input Method (D19-3)

Copied to clipboard

Chenchen Ding, Masao Utiyama, Eiichiro Sumita

Challenge:	MY-AKKHARA is a method used to input Burmese texts encoded in the Unicode standard, based on commonly accepted Latin transcription.
Approach:	They propose a method to input Burmese texts encoded in the Unicode standard based on 26 lowercase Latin letters and 26 uppercase Latin keys as shortcuts for lowercase letters.
Outcome:	The proposed method can input arbitrary Burmese strings with 26 lowercase Latin letters and 26 uppercase Latin characters on a QWERTY keyboard.

What Works and Doesn’t Work, A Deep Decoder for Neural Machine Translation (2022.findings-acl)

Copied to clipboard

Zuchao Li, Yiran Wang, Masao Utiyama, Eiichiro Sumita, Hai Zhao, Taro Watanabe

Language Models

Challenge:	Deep learning has demonstrated performance advantages in a wide range of natural language processing tasks.
Approach:	They propose to deepen the decoder layer in a Transformer model to reduce the difficulty of deep learning.
Outcome:	The proposed method can deepen the model on both the encoder and decoder at the same time, resulting in a deeper model and improved performance.

YANMTT: Yet Another Neural Machine Translation Toolkit (2023.acl-demo)

Copied to clipboard

Raj Dabre, Diptesh Kanojia, Chinmay Sawant, Eiichiro Sumita

Dynamic Sentence Sampling for Efficient Training of Neural Machine Translation (P18-2)

Challenge:	Neural machine translation (NMT) is an end-to-end approach that provides stateof-the-art results for a variety of language pairs.
Approach:	They propose to build an open-source neural machine translation toolkit on top of HuggingFace's Transformers library and use it for pre-training and fine-tuning sequence-to-sequence models.
Outcome:	The proposed toolkit is built on top of the HuggingFace Transformers library and provides advanced features such as document/multi-source NMT, simultaneous NMT and mixtures-of-experts.

Copied to clipboard

Rui Wang, Masao Utiyama, Eiichiro Sumita

FeatureBART: Feature Based Sequence-to-Sequence Pre-Training for Low-Resource NMT (2022.coling-1)

Challenge:	Existing methods to train neural machine translation (NMT) use a fixed training procedure where each sentence is sampled once during each epoch.
Approach:	They propose to dynamically sample sentences to accelerate NMT training . a weight is assigned to each sentence based on the measured difference between training costs of two iterations.
Outcome:	Empirical results show that the proposed method can significantly accelerate training and improve NMT performance.

Copied to clipboard

Abhisek Chakrabarty, Raj Dabre, Chenchen Ding, Hideki Tanaka, Masao Utiyama, Eiichiro Sumita

Supervised and Unsupervised Machine Translation for Myanmar-English and Khmer-English (D19-52)

Challenge:	FeatureBART is a linguistically motivated sequence-to-sequence monolingual pre-training strategy . syntactic features such as lemma, part-of-speech and dependency labels are incorporated into the pre-trained model .
Approach:	They propose a linguistically motivated sequence-to-sequence monolingual pre-training strategy that incorporates syntactic features into the framework.
Outcome:	The proposed model improves translation quality in bilingual and multilingual settings over models that do not use features.

Copied to clipboard

Benjamin Marie, Hour Kaing, Aye Myat Mon, Chenchen Ding, Atsushi Fujita, Masao Utiyama, Eiichiro Sumita

CytonMT: an Efficient Neural Machine Translation Open-source Toolkit Implemented in C++ (D18-2)

Challenge:	Using cleaned and normalized noisy monolingual data, supervised neural and statistical machine translation systems performed among the best for the four translation directions.
Approach:	They present supervised and unsupervised machine translation systems for the WAT2019 Myanmar-English and Khmer-English translation tasks.
Outcome:	The proposed systems performed among the best for the four translation directions.

Copied to clipboard

Xiaolin Wang, Masao Utiyama, Eiichiro Sumita

Subset Retrieval Nearest Neighbor Machine Translation (2023.acl-long)

Challenge:	Neural machine translation (NMT) has made remarkable progress over the past few years.
Approach:	They propose to use C++ and NVIDIA’s GPU-accelerated libraries to build an open-source neural machine translation toolkit called CytonMT.
Outcome:	The proposed toolkit accelerates the training speed by 64.5% to 110.8% on neural networks of various sizes, and achieves competitive translation quality.

Copied to clipboard

Hiroyuki Deguchi, Taro Watanabe, Yusuke Matsui, Masao Utiyama, Hideki Tanaka, Eiichiro Sumita

Restricted or Not: A General Training Framework for Neural Machine Translation (2022.acl-srw)

Challenge:	k-nearest-neighbor machine translation (kNN-MT) is a new approach to improve NMT performance without additional training.
Approach:	They propose a method that integrates example-search into the decoding algorithm to improve neighbor token retrieval.
Outcome:	The proposed method achieves a speed-up of up to 132.2 times and an improvement in BLEU score of up 1.6 compared with kNN-MT in the WMT’19 translation task and the domain adaptation tasks in De-En and En-Ja.

Copied to clipboard

Zuchao Li, Masao Utiyama, Eiichiro Sumita, Hai Zhao

Reference Language based Unsupervised Neural Machine Translation (2020.findings-emnlp)

Challenge:	Existing work imposes constraints on beam search decoding, which limits the concurrent processing ability of the model in deployment.
Approach:	They propose a general training framework that allows a model to support both restricted and unrestricted translations by adopting an additional auxiliary training process without constraining the decoding process.
Outcome:	The proposed training framework is tested on simulated and original benchmarks.

Copied to clipboard

Zuchao Li, Hai Zhao, Rui Wang, Masao Utiyama, Eiichiro Sumita

Pre-training via Leveraging Assisting Languages for Neural Machine Translation (2020.acl-srw)

Challenge:	Existing approaches to use a common language as an auxiliary for better translation have a long tradition in machine translation.
Approach:	They propose a reference language-based framework for unsupervised neural machine translation that uses only one auxiliary language as an auxiliary for better translation.
Outcome:	The proposed framework improves the quality of pivot translation over a baseline that uses only one auxiliary language.

Copied to clipboard

Haiyue Song, Raj Dabre, Zhuoyuan Mao, Fei Cheng, Sadao Kurohashi, Eiichiro Sumita

NICT’s participation to WAT 2019: Multilingualism and Multi-step Fine-Tuning for Low Resource NMT (D19-52)

Challenge:	Sequence-to-sequence (S2S) pre-training with large monolingual data is not always available for the languages of interest (LOI).
Approach:	They propose to use monolingual corpora of other languages to complement the scarce monolingual LOI by script mapping (Chinese to Japanese) . Using only Chinese and French monolinguals, they improve Japanese-English translation quality by up to 8.5 BLEU in low-resource scenarios.
Outcome:	The proposed approach improves Japanese-English translation quality by up to 8.5 BLEU in low-resource scenarios.

Copied to clipboard

Raj Dabre, Eiichiro Sumita

Multilingual Parallel Corpus for Global Communication Plan (L18-1)

Challenge:	In this paper, we describe our submissions for the following tasks: English–Tamil translation and Russian–Japanese translation.
Approach:	They propose to use multilingual domain adaptation and back-translation to improve translations in Russian–Japanese and English–Tamil.
Outcome:	The proposed techniques perform better in Russian–Japanese and English–Tamil translation tasks.

Copied to clipboard

Kenji Imamura, Eiichiro Sumita

MiSS: An Assistant for Multi-Style Simultaneous Translation (2021.emnlp-demo)

Challenge:	In this paper, we introduce the Global Communication Plan (GCP) Corpus . the corpus is sentence-aligned and covers ten languages, including many Asian languages .
Approach:	They introduce the Global Communication Plan (GCP) Corpus, a multilingual parallel corpus . it is sentence-aligned and covers ten languages, including many Asian languages .
Outcome:	The proposed corpus is sentence-aligned and covers ten languages, including many Asian languages.

Copied to clipboard

Zuchao Li, Kevin Parnow, Masao Utiyama, Eiichiro Sumita, Hai Zhao

Improving Low-Resource NMT through Relevance Based Linguistic Features Incorporation (2020.coling-main)

Challenge:	MiSS is a multi-style simultaneous translation assistant . it has five key features: high translation accuracy, simultaneous translation, flexibility, and measurable translation quality.
Approach:	They propose an assistant system for multi-style simultaneous translation that provides a complete translation experience for machine translation users.
Outcome:	The proposed system improves translation efficiency and performance by combining machine translation, grammatical error correction, and interactive edits.

Copied to clipboard

Abhisek Chakrabarty, Raj Dabre, Chenchen Ding, Masao Utiyama, Eiichiro Sumita

Incorporating Word Attention into Character-Based Word Segmentation (N19-1)

Challenge:	Existing studies on incorporating arbitrary syntactic information into neural machine translation (NMT) are lacking.
Approach:	They propose to integrate linguistic knowledge at different levels into neural machine translation framework to improve translation quality for language pairs with extremely limited data.
Outcome:	The proposed methods improve translation quality for all tasks by 3.09 BLEU points . the proposed methods are based on two different approaches .

Copied to clipboard

Shohei Higashiyama, Masao Utiyama, Eiichiro Sumita, Masao Ideuchi, Yoshiaki Oida, Yohei Sakamoto, Isaac Okada

Syntactic Text Processing

Challenge:	Word segmentation models are used to minimize the effort in feature engineering.
Approach:	They propose a character-based model that learns the importance of multiple candidate words for a corresponding character on the basis of an attention mechanism and makes use of it for segmentation decisions.
Outcome:	The proposed model outperforms the state-of-the-art models on Japanese and Chinese benchmark datasets.

Exploring Recombination for Efficient Decoding of Neural Machine Translation (D18-1)

Copied to clipboard

Zhisong Zhang, Rui Wang, Masao Utiyama, Eiichiro Sumita, Hai Zhao

English-Myanmar Supervised and Unsupervised NMT: NICT’s Machine Translation Systems at WAT-2019 (D19-52)

Challenge:	Neural Machine Translation (NMT) decoder captures features of entire prediction history . some partial hypotheses with different prefixes will be regarded differently no matter how similar they are .
Approach:	They propose a method that uses a n-gram suffix to adapt it to beam search decoding.
Outcome:	The proposed method can obtain similar translation quality with a smaller beam size, making it more efficient.

Copied to clipboard

Rui Wang, Haipeng Sun, Kehai Chen, Chenchen Ding, Masao Utiyama, Eiichiro Sumita

Robust Unsupervised Neural Machine Translation with Adversarial Denoising Training (2020.coling-main)

Challenge:	NICT participated in the 6th Workshop on Asian Translation (WAT-2019) shared translation task, specifically Myanmar (My) - English task in both translation directions.
Approach:	They present the participation of the NICT in the 6th Workshop on Asian Translation (WAT-2019) shared translation task, specifically Myanmar (Burmese) - English task in both translation directions.
Outcome:	The proposed systems perform the third in English-to-Myanmar and the second in Myanmar-to English according to BLEU score.

Copied to clipboard

Haipeng Sun, Rui Wang, Kehai Chen, Xugang Lu, Masao Utiyama, Eiichiro Sumita, Tiejun Zhao

Responsible & Trustworthy NLP

Challenge:	Unsupervised neural machine translation (UNMT) has attracted great interest in the machine translation community.
Approach:	They propose to explicitly take noisy data into consideration to improve the robustness of UNMT based systems.
Outcome:	The proposed methods significantly improved the robustness of the conventional UNMT systems in noisy scenarios.

Neural Machine Translation with Reordering Embeddings (P19-1)

Copied to clipboard

Kehai Chen, Rui Wang, Masao Utiyama, Eiichiro Sumita

Semantic Text Processing

Challenge:	Existing work exploits the reordering information in neural machine translation . experimental results show that the proposed methods can significantly improve the performance of the transformer translation system.
Approach:	They propose a reordering mechanism to learn the re ordering embedding of a word based on contextual information and stack them together with self-attention networks to learn sentence representation for machine translation.
Outcome:	The proposed method improves translation performance on English-to-German, NIST Chinese-to English, and WAT Japanese-toEnglish translation tasks.

Content Word Aware Neural Machine Translation (2020.acl-main)

Copied to clipboard

Kehai Chen, Rui Wang, Masao Utiyama, Eiichiro Sumita

Japanese-to-English Simultaneous Dubbing Prototype (2023.acl-demo)

Challenge:	Empirical results show that NMT does not consider word importance when predicting translations.
Approach:	They propose a content word-aware NMT model that exploits the results of translation using a sequence of content words learned by a simple content word recognition method.
Outcome:	Empirical results show that the proposed model improves translation performance . it uses word frequency information to distinguish between content and function words .

Copied to clipboard

Xiaolin Wang, Masao Utiyama, Eiichiro Sumita

Multimodality

Challenge:	Our system translates and replaces the original speech of a live video stream in a simultaneous manner.
Approach:	They propose a simultaneous dubbing prototype that translates and replaces the original speech of a live video stream in a simultaneous manner.
Outcome:	The proposed system achieves a low average latency of 11.90 seconds and meets a smoothness criterion.

Synchronous Refinement for Neural Machine Translation (2022.findings-acl)

Copied to clipboard

Kehai Chen, Masao Utiyama, Eiichiro Sumita, Rui Wang, Min Zhang

Language Models

Challenge:	Existing approaches to decode target sentences face a one-pass issue . generated wrong words are added to the historical context to affect the generation of subsequent target words, which hinders the performance of machine translation.
Approach:	They propose a synchronous refinement method to revise potential errors in the generated words by considering part of the target future context.
Outcome:	The proposed method can refine generated target words and generate the next target word synchronously.

A Myanmar (Burmese)-English Named Entity Transliteration Dictionary (2020.lrec-1)

Copied to clipboard

Aye Myat Mon, Chenchen Ding, Hour Kaing, Khin Mar Soe, Masao Utiyama, Eiichiro Sumita

Challenge:	Currently, there are no data available for the transcription of borrowed English words in Myanmar . lack of resources is a problem for many understudied languages .
Approach:	They construct a dictionary of Myanmar-English transliteration instances using a CC BY-NC-SA license.
Outcome:	The proposed model outperforms the statistical model significantly on the character level.

Bilingual Subword Segmentation for Neural Machine Translation (2020.coling-main)

Copied to clipboard

Hiroyuki Deguchi, Masao Utiyama, Akihiro Tamura, Takashi Ninomiya, Eiichiro Sumita

Intermediate Self-supervised Learning for Machine Translation Quality Estimation (2020.coling-main)

Challenge:	Existing subword segmentation methods tokenize sentences without considering translation . proposed method could be more favorable to machine translation if it uses bilingual sentences .
Approach:	They propose a subword segmentation method that tokenizes sentences by using subword units induced from bilingual sentences.
Outcome:	The proposed method improves translation performance on translation tasks up to +0.81 BLEU.

Copied to clipboard

Raphael Rubino, Eiichiro Sumita

Unsupervised Neural Machine Translation with Universal Grammar (2021.emnlp-main)

Challenge:	Existing methods for machine translation quality estimation (QE) rely on annotated data.
Approach:	They propose a self-supervised learning task for machine translation (MT) that orients a pre-trained model towards the target task.
Outcome:	The proposed method outperforms existing methods on English-to-German and English- to-Russian translation directions and is comparable to existing models.

Copied to clipboard

Zuchao Li, Masao Utiyama, Eiichiro Sumita, Hai Zhao

Forest-Based Neural Machine Translation (P18-1)

Challenge:	Unsupervised machine translation relies on parallel corpora for training, but performance still lags behind traditional supervised machine translators.
Approach:	They propose to leverage shared grammar clues to provide more explicit language parallel signals to enhance the training of unsupervised machine translation models.
Outcome:	The proposed models improve on a common language pair training task in English and german, and use embedding alignments and pretrained language models to synthesize pseudo parallel corpora.

Copied to clipboard

Chunpeng Ma, Akihiro Tamura, Masao Utiyama, Tiejun Zhao, Eiichiro Sumita

Knowledge Distillation for Multilingual Unsupervised Neural Machine Translation (2020.acl-main)

Challenge:	Compared with string-to-string systems, tree-based NMT methods use more syntactic information and can incorporate prior knowledge.
Approach:	They propose a tree-based neural machine translation method that translates a linearized packed forest under a simple sequence-to-sequence framework.
Outcome:	The proposed method outperforms tree-based approaches in the BLEU score of the proposed model.

Copied to clipboard

Haipeng Sun, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita, Tiejun Zhao

Responsible & Trustworthy NLP

Challenge:	Unsupervised neural machine translation (UNMT) can only translate between a single language pair and cannot produce translation results for multiple language pairs at the same time.
Approach:	They propose a method to translate between 13 languages using a single encoder and a decoder . they propose two knowledge distillation methods to further enhance multilingual UNMT performance .
Outcome:	The proposed method improves translation performance for all languages using multilingual data.

Self-Training for Unsupervised Neural Machine Translation in Unbalanced Training Data Scenarios (2021.naacl-main)

Copied to clipboard

Haipeng Sun, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita, Tiejun Zhao

User-Generated Text Corpus for Evaluating Japanese Morphological Analysis and Lexical Normalization (2021.naacl-main)

Challenge:	Existing methods that use monolingual corpora for translation are not suitable for low-resource languages such as Estonian.
Approach:	They propose unsupervised neural machine translation (UNMT) that relies on monolingual corpora to train a robust UNMT system and improve its performance.
Outcome:	The proposed methods outperform conventional UNMT systems on several language pairs.

Copied to clipboard

Shohei Higashiyama, Masao Utiyama, Taro Watanabe, Eiichiro Sumita

Syntactic Text Processing

Challenge:	Morphological analysis (MA) and lexical normalization (LN) are important tasks for Japanese user-generated text.
Approach:	They construct a publicly available Japanese UGT corpus annotated with morphological and normalization information.
Outcome:	The proposed corpus shows low performance for non-general words and non-standard forms . morphological analysis is an important task in Japanese user-generated text .

Unsupervised Bilingual Word Embedding Agreement for Unsupervised Neural Machine Translation (P19-1)

Copied to clipboard

Haipeng Sun, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita, Tiejun Zhao

Long Warm-up and Self-Training: Training Strategies of NICT-2 NMT System at WAT-2019 (D19-52)

Challenge:	Unsupervised bilingual word embedding (UBWE) has helped unsupervised neural machine translation (UNMT) achieve remarkable results in several language pairs.
Approach:	They propose two methods that train UNMT with UBWE agreement . they propose to use UBwe to initialize word embedding in UNMT .
Outcome:	The proposed methods outperform conventional methods on several language pairs.

Copied to clipboard

Kenji Imamura, Eiichiro Sumita