Papers by Masao Utiyama
A Three-Parameter Rank-Frequency Relation in Natural Languages (2020.acl-main)
Copied to clipboard
| Challenge: | Existing empirical law to form rank-frequency relation in textual data is Zipf's/power law . |
| Approach: | They propose a rank-frequency relation that follows f r-(r+)- in textual data. |
| Outcome: | The proposed formulation is the power law when =0 and the Zipf–Mandelbrot law when=1 . |
Centroid-Based Efficient Minimum Bayes Risk Decoding (2024.findings-acl)
Copied to clipboard
| Challenge: | Minimum Bayes risk (MBR) decoding requires quadratic time since it computes the expected score between a translation hypothesis and all reference translations. |
| Approach: | They propose a centroid-based MBR decoding method that clusters the translations in the feature space and calculates the expected score using the centroids of each cluster. |
| Outcome: | The proposed method outperforms vanilla MBR decoding in translation quality by up to 0.5 COMET in the WMT’22 EnJa, EnDe, EnZh, and WMT'23 Enja translation tasks. |
Recurrent Positional Embedding for Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Existing translation systems that use positional embeddings only encode static order dependencies based on discrete numerical information, which may hinder the improvement of translation capacity. |
| Approach: | They propose a recurrent positional embedding approach based on word vectors that are learned by a neural network and integrated into existing multi-head self-attention models. |
| Outcome: | The proposed approach improves translation performance over the state-of-the-art Transformer baseline in English-to-German and NIST Chinese-to English translation tasks. |
Improving Neural Machine Translation with Neural Syntactic Distance (N19-1)
Copied to clipboard
| Challenge: | Neural syntactic distance (NSD) is used to represent constituent trees using a sequence whose length is identical to the number of words in the sentence. |
| Approach: | They propose five strategies to improve NMT with explicit use of syntactic information . et al., 2014) propose a set of five strategies that incorporate syntastic information into the encoder and/or decoder of the baseline model. |
| Outcome: | The proposed strategies improve translation performance of the baseline model (+2.1 (En–Ja), +1.3 (Ja–En), +1.2 (En-Ch), and +1.0 (Ch–En) BLEU. |
Sentence-Level Agreement for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Empirical results show that a sentence-level agreement module can significantly improve the performance of neural machine translation (NMT) |
| Approach: | They propose a sentence-level agreement module to minimize the difference between the representation of source and target sentences. |
| Outcome: | Empirical results show the proposed agreement module significantly improves translation performance. |
Smoothing Dialogue States for Open Conversational Machine Reading (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies train independent or pipeline systems for the two subtasks but are trivial by using hard-label decisions to activate question generation. |
| Approach: | They propose a method to smooth two dialogue states in one decoder and bridge decision making and question generation to provide a richer dialogue state reference. |
| Outcome: | The proposed method achieves state-of-the-art on the OR-ShARC dataset. |
MY-AKKHARA: A Romanization-based Burmese (Myanmar) Input Method (D19-3)
Copied to clipboard
| Challenge: | MY-AKKHARA is a method used to input Burmese texts encoded in the Unicode standard, based on commonly accepted Latin transcription. |
| Approach: | They propose a method to input Burmese texts encoded in the Unicode standard based on 26 lowercase Latin letters and 26 uppercase Latin keys as shortcuts for lowercase letters. |
| Outcome: | The proposed method can input arbitrary Burmese strings with 26 lowercase Latin letters and 26 uppercase Latin characters on a QWERTY keyboard. |
What Works and Doesn’t Work, A Deep Decoder for Neural Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Deep learning has demonstrated performance advantages in a wide range of natural language processing tasks. |
| Approach: | They propose to deepen the decoder layer in a Transformer model to reduce the difficulty of deep learning. |
| Outcome: | The proposed method can deepen the model on both the encoder and decoder at the same time, resulting in a deeper model and improved performance. |
PrahokBART: A Pre-trained Sequence-to-Sequence Model for Khmer Natural Language Generation (2025.coling-main)
Copied to clipboard
| Challenge: | Pre-trained sequence-to-sequence models are typically pretrained on extensive raw text corpora and fine-tuned on task-specific data. |
| Approach: | They introduce a pre-trained sequence-to-sequence model trained from scratch for Khmer using carefully curated Khmer and English corpora. |
| Outcome: | The proposed model outperforms existing models on three generative tasks and is data-efficient and effective in enhancing performance across various natural language generation tasks. |
Dynamic Sentence Sampling for Efficient Training of Neural Machine Translation (P18-2)
Copied to clipboard
| Challenge: | Existing methods to train neural machine translation (NMT) use a fixed training procedure where each sentence is sampled once during each epoch. |
| Approach: | They propose to dynamically sample sentences to accelerate NMT training . a weight is assigned to each sentence based on the measured difference between training costs of two iterations. |
| Outcome: | Empirical results show that the proposed method can significantly accelerate training and improve NMT performance. |
FeatureBART: Feature Based Sequence-to-Sequence Pre-Training for Low-Resource NMT (2022.coling-1)
Copied to clipboard
| Challenge: | FeatureBART is a linguistically motivated sequence-to-sequence monolingual pre-training strategy . syntactic features such as lemma, part-of-speech and dependency labels are incorporated into the pre-trained model . |
| Approach: | They propose a linguistically motivated sequence-to-sequence monolingual pre-training strategy that incorporates syntactic features into the framework. |
| Outcome: | The proposed model improves translation quality in bilingual and multilingual settings over models that do not use features. |
Registering Source Tokens to Target Language Spaces in Multilingual Neural Machine Translation (2025.acl-long)
Copied to clipboard
| Challenge: | Multilingual neural machine translation (MNMT) aims for arbitrary translations across multiple languages. |
| Approach: | They propose a method that inserts a set of tokens specifying the target language into the input sequence between the source and target tokens. |
| Outcome: | The proposed method outperforms existing models on a large-scale benchmark. |
Supervised and Unsupervised Machine Translation for Myanmar-English and Khmer-English (D19-52)
Copied to clipboard
Benjamin Marie, Hour Kaing, Aye Myat Mon, Chenchen Ding, Atsushi Fujita, Masao Utiyama, Eiichiro Sumita
| Challenge: | Using cleaned and normalized noisy monolingual data, supervised neural and statistical machine translation systems performed among the best for the four translation directions. |
| Approach: | They present supervised and unsupervised machine translation systems for the WAT2019 Myanmar-English and Khmer-English translation tasks. |
| Outcome: | The proposed systems performed among the best for the four translation directions. |
CytonMT: an Efficient Neural Machine Translation Open-source Toolkit Implemented in C++ (D18-2)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) has made remarkable progress over the past few years. |
| Approach: | They propose to use C++ and NVIDIA’s GPU-accelerated libraries to build an open-source neural machine translation toolkit called CytonMT. |
| Outcome: | The proposed toolkit accelerates the training speed by 64.5% to 110.8% on neural networks of various sizes, and achieves competitive translation quality. |
Subset Retrieval Nearest Neighbor Machine Translation (2023.acl-long)
Copied to clipboard
| Challenge: | k-nearest-neighbor machine translation (kNN-MT) is a new approach to improve NMT performance without additional training. |
| Approach: | They propose a method that integrates example-search into the decoding algorithm to improve neighbor token retrieval. |
| Outcome: | The proposed method achieves a speed-up of up to 132.2 times and an improvement in BLEU score of up 1.6 compared with kNN-MT in the WMT’19 translation task and the domain adaptation tasks in De-En and En-Ja. |
Restricted or Not: A General Training Framework for Neural Machine Translation (2022.acl-srw)
Copied to clipboard
| Challenge: | Existing work imposes constraints on beam search decoding, which limits the concurrent processing ability of the model in deployment. |
| Approach: | They propose a general training framework that allows a model to support both restricted and unrestricted translations by adopting an additional auxiliary training process without constraining the decoding process. |
| Outcome: | The proposed training framework is tested on simulated and original benchmarks. |
Reference Language based Unsupervised Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to use a common language as an auxiliary for better translation have a long tradition in machine translation. |
| Approach: | They propose a reference language-based framework for unsupervised neural machine translation that uses only one auxiliary language as an auxiliary for better translation. |
| Outcome: | The proposed framework improves the quality of pivot translation over a baseline that uses only one auxiliary language. |
MiSS: An Assistant for Multi-Style Simultaneous Translation (2021.emnlp-demo)
Copied to clipboard
| Challenge: | MiSS is a multi-style simultaneous translation assistant . it has five key features: high translation accuracy, simultaneous translation, flexibility, and measurable translation quality. |
| Approach: | They propose an assistant system for multi-style simultaneous translation that provides a complete translation experience for machine translation users. |
| Outcome: | The proposed system improves translation efficiency and performance by combining machine translation, grammatical error correction, and interactive edits. |
Improving Low-Resource NMT through Relevance Based Linguistic Features Incorporation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing studies on incorporating arbitrary syntactic information into neural machine translation (NMT) are lacking. |
| Approach: | They propose to integrate linguistic knowledge at different levels into neural machine translation framework to improve translation quality for language pairs with extremely limited data. |
| Outcome: | The proposed methods improve translation quality for all tasks by 3.09 BLEU points . the proposed methods are based on two different approaches . |
Incorporating Word Attention into Character-Based Word Segmentation (N19-1)
Copied to clipboard
Shohei Higashiyama, Masao Utiyama, Eiichiro Sumita, Masao Ideuchi, Yoshiaki Oida, Yohei Sakamoto, Isaac Okada
| Challenge: | Word segmentation models are used to minimize the effort in feature engineering. |
| Approach: | They propose a character-based model that learns the importance of multiple candidate words for a corresponding character on the basis of an attention mechanism and makes use of it for segmentation decisions. |
| Outcome: | The proposed model outperforms the state-of-the-art models on Japanese and Chinese benchmark datasets. |
On Eliciting Syntax from Language Models via Hashing (2024.emnlp-main)
Copied to clipboard
| Challenge: | Unsupervised parsing aims to infer syntactic structure from raw text . despite its importance, advancements in this task have been slow . |
| Approach: | They propose to use unsupervised parsing to infer syntactic structure from raw text . they upgrade the bit-level CKY to first-order to encode lexicon and syntax . |
| Outcome: | The proposed method shows competitive performance on various datasets. |
Exploring Recombination for Efficient Decoding of Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) decoder captures features of entire prediction history . some partial hypotheses with different prefixes will be regarded differently no matter how similar they are . |
| Approach: | They propose a method that uses a n-gram suffix to adapt it to beam search decoding. |
| Outcome: | The proposed method can obtain similar translation quality with a smaller beam size, making it more efficient. |
English-Myanmar Supervised and Unsupervised NMT: NICT’s Machine Translation Systems at WAT-2019 (D19-52)
Copied to clipboard
| Challenge: | NICT participated in the 6th Workshop on Asian Translation (WAT-2019) shared translation task, specifically Myanmar (My) - English task in both translation directions. |
| Approach: | They present the participation of the NICT in the 6th Workshop on Asian Translation (WAT-2019) shared translation task, specifically Myanmar (Burmese) - English task in both translation directions. |
| Outcome: | The proposed systems perform the third in English-to-Myanmar and the second in Myanmar-to English according to BLEU score. |
Robust Unsupervised Neural Machine Translation with Adversarial Denoising Training (2020.coling-main)
Copied to clipboard
| Challenge: | Unsupervised neural machine translation (UNMT) has attracted great interest in the machine translation community. |
| Approach: | They propose to explicitly take noisy data into consideration to improve the robustness of UNMT based systems. |
| Outcome: | The proposed methods significantly improved the robustness of the conventional UNMT systems in noisy scenarios. |
Neural Machine Translation with Reordering Embeddings (P19-1)
Copied to clipboard
| Challenge: | Existing work exploits the reordering information in neural machine translation . experimental results show that the proposed methods can significantly improve the performance of the transformer translation system. |
| Approach: | They propose a reordering mechanism to learn the re ordering embedding of a word based on contextual information and stack them together with self-attention networks to learn sentence representation for machine translation. |
| Outcome: | The proposed method improves translation performance on English-to-German, NIST Chinese-to English, and WAT Japanese-toEnglish translation tasks. |
Content Word Aware Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Empirical results show that NMT does not consider word importance when predicting translations. |
| Approach: | They propose a content word-aware NMT model that exploits the results of translation using a sequence of content words learned by a simple content word recognition method. |
| Outcome: | Empirical results show that the proposed model improves translation performance . it uses word frequency information to distinguish between content and function words . |
Japanese-to-English Simultaneous Dubbing Prototype (2023.acl-demo)
Copied to clipboard
| Challenge: | Our system translates and replaces the original speech of a live video stream in a simultaneous manner. |
| Approach: | They propose a simultaneous dubbing prototype that translates and replaces the original speech of a live video stream in a simultaneous manner. |
| Outcome: | The proposed system achieves a low average latency of 11.90 seconds and meets a smoothness criterion. |
Comprehensive Evaluation on Lexical Normalization: Boundary-Aware Approaches for Unsegmented Languages (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Lexical normalization research has sought to tackle the challenge of processing informal expressions in user-generated text. |
| Approach: | They focus on Japanese normalization and developing methods based on state-of-the-art pre-trained models . |
| Outcome: | The proposed methods achieve high accuracy and efficiency across multiple evaluation perspectives. |
Synchronous Refinement for Neural Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to decode target sentences face a one-pass issue . generated wrong words are added to the historical context to affect the generation of subsequent target words, which hinders the performance of machine translation. |
| Approach: | They propose a synchronous refinement method to revise potential errors in the generated words by considering part of the target future context. |
| Outcome: | The proposed method can refine generated target words and generate the next target word synchronously. |
A Myanmar (Burmese)-English Named Entity Transliteration Dictionary (2020.lrec-1)
Copied to clipboard
| Challenge: | Currently, there are no data available for the transcription of borrowed English words in Myanmar . lack of resources is a problem for many understudied languages . |
| Approach: | They construct a dictionary of Myanmar-English transliteration instances using a CC BY-NC-SA license. |
| Outcome: | The proposed model outperforms the statistical model significantly on the character level. |
Robust Neural Machine Translation for Abugidas by Glyph Perturbation (2024.eacl-short)
Copied to clipboard
| Challenge: | Neural machine translation systems are vulnerable when trained on limited data. |
| Approach: | They propose to add noise to the training phase to increase robustness of NMT systems trained on limited data. |
| Outcome: | The proposed training strategy overcomes noise and improves robustness for low-resource tasks for abugida glyphs. |
Bilingual Subword Segmentation for Neural Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing subword segmentation methods tokenize sentences without considering translation . proposed method could be more favorable to machine translation if it uses bilingual sentences . |
| Approach: | They propose a subword segmentation method that tokenizes sentences by using subword units induced from bilingual sentences. |
| Outcome: | The proposed method improves translation performance on translation tasks up to +0.81 BLEU. |
Unsupervised Neural Machine Translation with Universal Grammar (2021.emnlp-main)
Copied to clipboard
| Challenge: | Unsupervised machine translation relies on parallel corpora for training, but performance still lags behind traditional supervised machine translators. |
| Approach: | They propose to leverage shared grammar clues to provide more explicit language parallel signals to enhance the training of unsupervised machine translation models. |
| Outcome: | The proposed models improve on a common language pair training task in English and german, and use embedding alignments and pretrained language models to synthesize pseudo parallel corpora. |
To be Continuous, or to be Discrete, Those are Bits of Questions (2024.acl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that binary representations preserve information on the output side, but they are not available for continuous representations. |
| Approach: | They propose to extend the contrastive hashing method to output binary labels instead of continuous vectors. |
| Outcome: | The proposed representation preserves structural information along with label information . it achieves competitive performance on various structured prediction tasks . |
Forest-Based Neural Machine Translation (P18-1)
Copied to clipboard
| Challenge: | Compared with string-to-string systems, tree-based NMT methods use more syntactic information and can incorporate prior knowledge. |
| Approach: | They propose a tree-based neural machine translation method that translates a linearized packed forest under a simple sequence-to-sequence framework. |
| Outcome: | The proposed method outperforms tree-based approaches in the BLEU score of the proposed model. |
Knowledge Distillation for Multilingual Unsupervised Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Unsupervised neural machine translation (UNMT) can only translate between a single language pair and cannot produce translation results for multiple language pairs at the same time. |
| Approach: | They propose a method to translate between 13 languages using a single encoder and a decoder . they propose two knowledge distillation methods to further enhance multilingual UNMT performance . |
| Outcome: | The proposed method improves translation performance for all languages using multilingual data. |
Self-Training for Unsupervised Neural Machine Translation in Unbalanced Training Data Scenarios (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods that use monolingual corpora for translation are not suitable for low-resource languages such as Estonian. |
| Approach: | They propose unsupervised neural machine translation (UNMT) that relies on monolingual corpora to train a robust UNMT system and improve its performance. |
| Outcome: | The proposed methods outperform conventional UNMT systems on several language pairs. |
User-Generated Text Corpus for Evaluating Japanese Morphological Analysis and Lexical Normalization (2021.naacl-main)
Copied to clipboard
| Challenge: | Morphological analysis (MA) and lexical normalization (LN) are important tasks for Japanese user-generated text. |
| Approach: | They construct a publicly available Japanese UGT corpus annotated with morphological and normalization information. |
| Outcome: | The proposed corpus shows low performance for non-general words and non-standard forms . morphological analysis is an important task in Japanese user-generated text . |
Unsupervised Bilingual Word Embedding Agreement for Unsupervised Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Unsupervised bilingual word embedding (UBWE) has helped unsupervised neural machine translation (UNMT) achieve remarkable results in several language pairs. |
| Approach: | They propose two methods that train UNMT with UBWE agreement . they propose to use UBwe to initialize word embedding in UNMT . |
| Outcome: | The proposed methods outperform conventional methods on several language pairs. |
Simplified Abugidas (P18-2)
Copied to clipboard
| Challenge: | Abugidas are writing systems where consonant letters represent syllables with a default vowel and other vowels are denoted by diacritics. |
| Approach: | They investigated the feasibility of recovering the original text written in an abugida after omitting subordinate diacritics and merging consonant letters with similar phonetic values. |
| Outcome: | The proposed method recovers the original text written in an abugida with 94% - 97% accuracy at the top-1 level and 98% - 99% at the bottom-4 level even after omitting most diacritics and merging the remaining 30 - 50 characters into 21 graphemes. |
Guiding Neural Machine Translation with Retrieved Translation Pieces (N18-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) has trouble with lowfrequency words or phrases and generalizing across domains. |
| Approach: | They propose a method for recalling low-frequency words and phrases into neural machine translation by retrieving n-grams from a search engine and incorporating them into the decoding process. |
| Outcome: | The proposed method improves translation results up to 6 BLEU points on three narrow domain translation tasks where repetitiveness of the target sentences is particularly salient. |