Papers by Yosuke Kishinami
Revisiting Non-Verbatim Memorization in Large Language Models: The Role of Entity Surface Forms (2026.acl-long)
Copied to clipboard
Yuto Nishida, Naoki Shikoda, Yosuke Kishinami, Ryo Fujii, Makoto Morishita, Hidetaka Kamigaito, Taro Watanabe
| Challenge: | Entity-based QA is a common framework for analyzing non-verbatim memorization, but typically query each entity using a single canonical surface form. |
| Approach: | They propose a dataset that pairs Wikidata factual triples with categorized entity surface forms . they examine surface-conditioned factual memorization and find that prediction outcomes change when only the entity surface form is changed. |
| Outcome: | The proposed dataset shows that large language models memorize factual knowledge when only the subject entity surface form is changed. |
Target-Guided Open-Domain Conversation Planning (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies on goal-oriented conversational tasks lack planning . prior studies on this topic have focused on generating proactive behavior in agents . |
| Approach: | They propose a task to evaluate whether neural conversational agents have goal-oriented conversation planning abilities. |
| Outcome: | The proposed task evaluates whether neural conversational agents have goal-oriented conversation planning abilities. |
Bipartite-play Dialogue Collection for Practical Automatic Evaluation of Dialogue Systems (2022.aacl-srw)
Copied to clipboard
| Challenge: | Existing methods for dialogue system evaluation are inefficient and time-consuming. |
| Approach: | They propose a dialogue collection method for automating dialogue system evaluation using bipartite-play method . authors propose constructing a better automatic evaluation method which is reproducible and low cost . |
| Outcome: | The proposed method correlates strongly with human subjectivity and human evaluation. |