Papers by Yoshimasa Tsuruoka
EASE: Entity-Aware Contrastive Learning of Sentence Embedding (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for learning sentence embeddings are fine-tuning general-purpose pretrained models with a particular training supervision. |
| Approach: | They propose a method for learning sentence embeddings via contrastive learning between sentences and related entities. |
| Outcome: | The proposed method outperforms baseline methods in multilingual settings on a variety of tasks. |
NeoAMT: Neologism-Aware Agentic Machine Translation with Reinforcement Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Neologism-aware machine translation aims to translate source sentences containing neologismes into target languages. |
| Approach: | They propose an agentic framework for neologism-aware machine translation equipped with a Wiktionary-based search toolkit. |
| Outcome: | The proposed framework is based on a Wiktionary-based search toolkit and a dedicated dataset for neologism-aware machine translation. |
Word Alignment as Preference for Machine Translation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Hallucination and omission are a problem in machine translation because of an LLM's size and low-resource languages. |
| Approach: | They propose to use word alignment as preference to optimize an LLM-based MT model to mitigate hallucination and omission problems. |
| Outcome: | The proposed model is able to mitigate hallucination and omission by using word alignment as preference. |
Pretraining with Artificial Language: Studying Transferable Knowledge in Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that pretraining with an artificial language with nesting dependency structure provides some knowledge transferable to natural language. |
| Approach: | They propose to pretrain artificial languages with structural properties that mimic natural language and then test their performance on downstream tasks. |
| Outcome: | The proposed language models show strong performance across languages and languages. |
Accelerated Reinforcement Learning for Sentence Generation by Vocabulary Prediction (N19-1)
Copied to clipboard
| Challenge: | In reinforcement learning-based sentence generation, the large action space is often too computa-tionally demanding to be used with large training data. |
| Approach: | They propose to reduce the action space by using dynamic vocabulary prediction to generate a fixed-size small vocabulary for each input to generate its target sentence. |
| Outcome: | The proposed method achieves faster reinforcement learning (2.7x faster) with less GPU memory (2.3x less) and more rewards with fewer iterations of supervised pre-training. |
Leveraging Multi-lingual Positive Instances in Contrastive Learning to Improve Sentence Embedding (2024.eacl-long)
Copied to clipboard
| Challenge: | Recent trends in learning monolingual and multilingual sentence embeddings are based on contrastive learning (CL) among an anchor, one positive and multiple negative instances. |
| Approach: | They propose to leverage multiple positives to improve learning of multilingual sentence embeddings by using an anchor, one positive, and multiple negative instances. |
| Outcome: | The proposed approach improves retrieval, semantic similarity, and classification performance on unseen languages. |
mLUKE: The Power of Entity Representations in Multilingual Pretrained Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for improving multilingual models only use entity information in pretraining and do not explicitly use entities in downstream tasks. |
| Approach: | They propose to leverage Wikipedia entity representations for downstream tasks . they train a multilingual language model with 24 languages with entity representation . |
| Outcome: | The proposed model outperforms word-based models in cross-lingual transfer tasks. |
Revisiting the Context Window for Cross-lingual Word Embeddings (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to mapping-based cross-lingual word embeddings are based on the assumption that the source and target embeddable spaces are structurally similar. |
| Approach: | They propose to use different context windows to evaluate bilingual word embeddings in various languages, domains, and tasks. |
| Outcome: | The size of both the source and target window improves bilingual lexicon induction, especially on frequent nouns. |
Using Semantic Similarity as Reward for Reinforcement Learning in Sentence Generation (P19-2)
Copied to clipboard
| Challenge: | Existing models for sentence generation use cross-entropy loss as the loss function . however, cross-etropy is unable to evaluate sentences as a whole and lacks flexibility . et al., 2018: a novel approach to improve sentence generation models . |
| Approach: | They propose a method to train a model using estimated semantic similarity between output and reference sentences to alleviate cross-entropy loss problems. |
| Outcome: | The proposed model improves the BLEU scores from the baseline LSTM NMT model. |
WSPAlign: Word Alignment Pre-training via Large-Scale Weakly Supervised Span Prediction (2023.acl-long)
Copied to clipboard
| Challenge: | Existing word alignment methods rely on manual data and lack generalization ability. |
| Approach: | They propose to use a weakly-supervised large-scale weakly supervised dataset for word alignment pre-training via span prediction to reduce the need for manual data. |
| Outcome: | The proposed method improves upon the best supervised baseline by 3.3 6.1 points in F1 and 1.5 6.1 point in AER. |
Data Augmentation with Unsupervised Machine Translation Improves the Structural Similarity of Cross-lingual Word Embeddings (2021.acl-srw)
Copied to clipboard
| Challenge: | Unsupervised cross-lingual word embedding methods learn a linear transformation matrix that maps two monolingual embeddable spaces that are separately trained with monolingual corpora. |
| Approach: | They propose a method that maps two monolingual embedding spaces that are separately trained with monolingual corpora using a pseudo-parallel corpus. |
| Outcome: | The proposed method outperforms other methods given the same amount of data and shows that using a pseudo-parallel corpus makes the source and target corpora (partially) parallel . |
Enhancing Cross-lingual Sentence Embedding for Low-resource Languages with Word Alignment (2024.findings-naacl)
Copied to clipboard
| Challenge: | Current approaches to obtain cross-lingual sentence embeddings rely on pre-trained language models that implicitly align the contextual representations of similar units of sentences in different languages. |
| Approach: | They propose a framework that explicitly aligns words between English and eight low-resource languages by using off-the-shelf word alignment models. |
| Outcome: | The proposed framework improves on the bitext retrieval task and in high-resource languages. |
Improving Word Alignment Using Semi-Supervised Learning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing word alignment methods rely on labeled data, but augmenting training with pseudo-labeled data improves performance. |
| Approach: | They propose a semi-supervised framework to improve word alignment methods . they use pseudo-labeled data from multilingual encoder models as word aligners . |
| Outcome: | The proposed framework outperforms the current state-of-the-art binary alignment method on word alignment datasets. |