Papers by Ryo Fujii
Revisiting Non-Verbatim Memorization in Large Language Models: The Role of Entity Surface Forms (2026.acl-long)
Copied to clipboard
Yuto Nishida, Naoki Shikoda, Yosuke Kishinami, Ryo Fujii, Makoto Morishita, Hidetaka Kamigaito, Taro Watanabe
| Challenge: | Entity-based QA is a common framework for analyzing non-verbatim memorization, but typically query each entity using a single canonical surface form. |
| Approach: | They propose a dataset that pairs Wikidata factual triples with categorized entity surface forms . they examine surface-conditioned factual memorization and find that prediction outcomes change when only the entity surface form is changed. |
| Outcome: | The proposed dataset shows that large language models memorize factual knowledge when only the subject entity surface form is changed. |
PheMT: A Phenomenon-wise Dataset for Machine Translation Robustness on User-Generated Contents (2020.coling-main)
Copied to clipboard
| Challenge: | Existing studies suggest that Neural Machine Translation still struggles with certain kinds of input with considerable noise, such as User-Generated Contents (UGC) on the Internet. |
| Approach: | They propose to evaluate the robustness of Neural Machine Translation models against specific linguistic phenomena in Japanese-English translation. |
| Outcome: | The proposed model can handle user-generated content (UGC) on the Internet, but it is difficult to translate clean inputs. |
TimeMachine-bench: A Benchmark for Evaluating Model Capabilities in Repository-Level Migration Tasks (2026.eacl-long)
Copied to clipboard
| Challenge: | Automated software engineering is a critical task of software engineers. |
| Approach: | They propose a benchmark to evaluate software migration in real-world Python projects. |
| Outcome: | The proposed benchmark consists of GitHub repositories whose tests fail in response to dependency updates. |