Papers by Yuto Nishida
Revisiting Non-Verbatim Memorization in Large Language Models: The Role of Entity Surface Forms (2026.acl-long)
Copied to clipboard
Yuto Nishida, Naoki Shikoda, Yosuke Kishinami, Ryo Fujii, Makoto Morishita, Hidetaka Kamigaito, Taro Watanabe
| Challenge: | Entity-based QA is a common framework for analyzing non-verbatim memorization, but typically query each entity using a single canonical surface form. |
| Approach: | They propose a dataset that pairs Wikidata factual triples with categorized entity surface forms . they examine surface-conditioned factual memorization and find that prediction outcomes change when only the entity surface form is changed. |
| Outcome: | The proposed dataset shows that large language models memorize factual knowledge when only the subject entity surface form is changed. |
How to Make the Most of LLMs’ Grammatical Knowledge for Acceptability Judgments (2025.naacl-long)
Copied to clipboard
Yusuke Ide, Yuto Nishida, Justin Vasselli, Miyu Oba, Yusuke Sakai, Hidetaka Kamigaito, Taro Watanabe
| Challenge: | Conventional approaches compare sentence probabilities directly, but large language models (LLMs) provide nuanced evaluation methods using prompts and templates. |
| Approach: | They propose to derive acceptability judgments from large language models using prompts and templates to comprehensively evaluate their grammatical knowledge. |
| Outcome: | The proposed methods excel in different linguistic phenomena, suggesting they access different aspects of the LLMs’ grammatical knowledge. |
Long-Tail Crisis in Nearest Neighbor Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Prior studies have shown that kNN-LM can retrieve long-tail contexts, leaving the model’s performance underexplored in estimating the probabilities of long-tailed target tokens. |
| Approach: | They investigate the behavior of kNN-LM on low-frequency tokens, examining prediction probability, retrieval accuracy, and token distribution in the datastore. |
| Outcome: | The proposed model improves the perplexity of given text by directly accessing a large datastore built from any text data during inference. |
Generating Diverse Translation with Perturbed kNN-MT (2024.eacl-srw)
Copied to clipboard
| Challenge: | Existing methods to generate multiple translation candidates do not address the overcorrection problem, which discourages the model from generating synonymous expressions and leans toward gold standards, reducing the diversity in the candidates. |
| Approach: | They propose to introduce perturbed k-nearest neighbor machine translation (kNN-MT) to generate more diverse translations. |
| Outcome: | The proposed methods significantly improve candidate diversity and control diversity by tuning the perturbation’s magnitude. |
Instability in Downstream Task Performance During LLM Pretraining (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a study of large language models shows that task scores fluctuate throughout training . |
| Approach: | They empirically analyze the stability of downstream task performance in an LLM . |
| Outcome: | The proposed methods improve performance stability without changes to the training procedure. |