Papers by Hiroaki Sugiyama
Time-Considerable Dialogue Models via Reranking by Time Dependency (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing dialogue models do not consider time information that humans are constantly aware of. |
| Approach: | They propose to categorize responses by their naturalness at different times and introduce a new metric to classify responses into categories. |
| Outcome: | The proposed model categorizes responses by their naturalness at different times and evaluates them subjectively. |
Let’s Put Ourselves in Sally’s Shoes: Shoes-of-Others Prefilling Improves Theory of Mind in Large Language Models (2026.findings-eacl)
Copied to clipboard
Kazutoshi Shinoda, Nobukatsu Hojo, Kyosuke Nishida, Yoshihiro Yamazaki, Keita Suzuki, Hiroaki Sugiyama, Kuniko Saito
| Challenge: | Existing methods for Theory of Mind (ToM) are specialized for inferring beliefs from contexts involving changes in the world state. |
| Approach: | They propose a method which makes fewer assumptions about contexts and is applicable to broader scenarios. |
| Outcome: | The proposed method makes fewer assumptions about contexts and is applicable to broader scenarios. |
Collection of Multimodal Dialog Data and Analysis of the Result of Annotation of Users’ Interest Level (L18-1)
Copied to clipboard
Masahiro Araki, Sayaka Tomimasu, Mikio Nakano, Kazunori Komatani, Shogo Okada, Shinya Fujie, Hiroaki Sugiyama
| Challenge: | a group of researchers is building a corpus for evaluating elements of multimodal dialogue systems. |
| Approach: | They propose to build a corpus for evaluating elements of the multimodal dialogue system . they use the Wizard of Oz method to record chat dialogue data between a human and a virtual agent . |
| Outcome: | The proposed method annotates chat dialogue data between a human and a virtual agent and measures their interest level in the data. |
Comparison of the Intimacy Process between Real and Acting-based Long-term Text Chats (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent open-domain chatbots can generate natural responses over multiple turns using large-scale language models, but they do not address the speaker intimacy process and thus cannot sustain natural dialogue over multiple days and weeks. |
| Approach: | They propose to train systems with multi-session chat data to simulate relationship-building between speakers. |
| Outcome: | The proposed multi-session chat data can simulate relationship-building between speakers but has not been tested in Japanese. |
Bipartite-play Dialogue Collection for Practical Automatic Evaluation of Dialogue Systems (2022.aacl-srw)
Copied to clipboard
| Challenge: | Existing methods for dialogue system evaluation are inefficient and time-consuming. |
| Approach: | They propose a dialogue collection method for automating dialogue system evaluation using bipartite-play method . authors propose constructing a better automatic evaluation method which is reproducible and low cost . |
| Outcome: | The proposed method correlates strongly with human subjectivity and human evaluation. |