Papers by Makoto Yamada
Large-scale similarity search with Optimal Transport (2023.emnlp-main)
Copied to clipboard
| Challenge: | Word mover's distance (WMD) is a powerful tool for comparing probability distributions in NLP. |
| Approach: | They propose a waterstein distance approximation that uses the L1 embedding method to find the k-nearest neighbors. |
| Outcome: | The proposed approximation performs comparable to the vanilla Wasserstein distance and can be computed three orders of magnitude faster than the vanilla waterstein distance. |
Computationally Efficient Wasserstein Loss for Structured Labels (2021.eacl-srw)
Copied to clipboard
| Challenge: | Existing approaches to estimate the probability distribution of labels are based on tree-Wasserstein distance. |
| Approach: | They propose a tree-Wasserstein distance regularized LDL algorithm for hierarchical text classification tasks. |
| Outcome: | The proposed method performs well on synthetic and real-world datasets and compares favorably with the Sinkhorn algorithm in terms of computation time and memory usage. |
Data Poisoning for In-context Learning (2025.findings-naacl)
Copied to clipboard
| Challenge: | In-context learning (ICL) has emerged as a capability of large language models (LLMs) but there is limited understanding of its vulnerability against data poisoning attacks. |
| Approach: | They propose an attack method that exploits ICL’s unique learning mechanisms by identifying discrete text perturbations that influence LLM hidden states. |
| Outcome: | The proposed attack method exploits ICL’s learning mechanisms by identifying discrete text perturbations that influence LLM hidden states. |
Transformer Dissection: An Unified Understanding for Transformer’s Attention via the Lens of Kernel (D19-1)
Copied to clipboard
| Challenge: | Transformer is a powerful architecture that achieves superior performance on various sequence learning tasks, including neural machine translation, language understanding, and sequence prediction. |
| Approach: | They propose a new formulation of attention via the lens of the kernel which allows us to understand individual components of Transformer's attention. |
| Outcome: | The proposed model outperforms existing models on language understanding and sequence prediction tasks and is more efficient than existing models. |
Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are susceptible to a type of attack known as jailbreaking, which misleads LLMs to output harmful contents. |
| Approach: | They propose to leverage hidden representations into existing jailbreak targets to move the attacks along the acceptance direction. |
| Outcome: | The proposed methods are validated using the objective of existing jailbreak attacks. |
A linear time approximation of Wasserstein distance with word embedding selection (2023.emnlp-main)
Copied to clipboard
| Challenge: | Wasserstein distance is a powerful method for measuring the dissimilarity between distributions and is used in natural language processing to measure dissimilarities between documents. |
| Approach: | They propose a method to combine feature selection and tree approximation of Wasserstein distance to handle high-dimensional problems. |
| Outcome: | The proposed method achieves high performance on document classification using word embeddings and word embeds. |
Learning Unsupervised Word Translations Without Adversaries (D18-1)
Copied to clipboard
| Challenge: | Current methods for word translation are based on adversarial models and suffer from instability and hyper-parameter sensitivity. |
| Approach: | They propose a statistical dependency-based approach to bilingual dictionary induction that is unsupervised and introduces no adversary. |
| Outcome: | The proposed method outperforms adversarial alternatives and is much easier to train. |