Papers by Ayuki Katayama
Reliability of Distribution Predictions by LLMs: Insights from Counterintuitive Pseudo-Distributions (2025.naacl-srw)
Copied to clipboard
Toma Suzuki, Ayuki Katayama, Seiji Gobara, Ryo Tsujimoto, Hibiki Nakatani, Kazuki Hayashi, Yusuke Sakai, Hidetaka Kamigaito, Taro Watanabe
| Challenge: | Recent studies highlight the use of Large Language Models (LLMs) for predicting response distributions as a cost-effective survey method. |
| Approach: | They examine whether LLMs can rationally estimate distributions when presented with explanations that are against commonsense. |
| Outcome: | The proposed models can rationally estimate distributions when presented with explanations that are against commonsense, but smaller or less human-optimized models follow explanations uncritically, compared to larger models that resist counterintuitive explanations by leveraging their pretraining-acquired knowledge. |