Papers by Yoichi Aoki
LLMs Faithfully and Iteratively Compute Answers During CoT: A Systematic Analysis With Multi-step Arithmetics (2026.findings-eacl)
Copied to clipboard
Keito Kudo, Yoichi Aoki, Tatsuki Kuribayashi, Shusaku Sone, Masaya Taniguchi, Ana Brassard, Keisuke Sakaguchi, Kentaro Inui
| Challenge: | Specifically, we examine when the LLMs’ answer is (pre)determined, especially before the CoT begins or after, and how strongly the information from CoT specifically has a causal effect on the final answer. |
| Approach: | They examine when the LLMs’ answer is (pre)determined, especially before the CoT begins or after, and how strongly the information from CoT specifically has a causal effect on the final answer. |
| Outcome: | The proposed model can generate reasoning chains while generating the reasoning chain on the fly. |
Empirical Investigation of Neural Symbolic Reasoning Strategies (2023.findings-eacl)
Copied to clipboard
Yoichi Aoki, Keito Kudo, Tatsuki Kuribayashi, Ana Brassard, Masashi Yoshikawa, Keisuke Sakaguchi, Kentaro Inui
| Challenge: | Neural reasoning accuracy improves when generating intermediate reasoning steps. |
| Approach: | They decompose the reasoning strategy w.r.t. step granularity and chaining strategy. |
| Outcome: | The proposed reasoning strategy significantly affects performance in a symbolic reasoning dataset. |
Do Deep Neural Networks Capture Compositionality in Arithmetic Reasoning? (2023.eacl-main)
Copied to clipboard
Keito Kudo, Yoichi Aoki, Tatsuki Kuribayashi, Ana Brassard, Masashi Yoshikawa, Keisuke Sakaguchi, Kentaro Inui
| Challenge: | Using a pre-trained dataset, we examine how well recent neural models capture compositionality in symbolic reasoning tasks. |
| Approach: | They propose a skill tree on compositionality that defines hierarchical levels of complexity along with three compositionality dimensions: systematicity, productivity, and substitutivity. |
| Outcome: | The proposed model struggled most with systematicity, performing poorly even with relatively simple compositions. |
How Well Do Vision Models Encode Diagram Attributes? (2024.acl-srw)
Copied to clipboard
Haruto Yoshida, Keito Kudo, Yoichi Aoki, Ryota Tanaka, Itsumi Saito, Keisuke Sakaguchi, Kentaro Inui
| Challenge: | Experimental results show vision models struggle to identify diagram attributes such as node colors and shapes, along with edge colors and connection patterns. |
| Approach: | They evaluated vision models and retrieving diagrams using text queries to determine how well they recognize diagram attributes and edge connection patterns. |
| Outcome: | The models can recognize node colors, shapes, and edge colors, but struggle to identify differences in edge connection patterns that play a pivotal role in the semantics of diagrams. |
First Heuristic Then Rational: Dynamic Use of Heuristics in Language Model Reasoning (2024.emnlp-main)
Copied to clipboard
Yoichi Aoki, Keito Kudo, Tatsuki Kuribayashi, Shusaku Sone, Masaya Taniguchi, Keisuke Sakaguchi, Kentaro Inui
| Challenge: | Explicit multi-step reasoning is widely adopted to improve the performance of language models. |
| Approach: | They propose a systematic reasoning strategy that LMs use to solve multi-step reasoning tasks. |
| Outcome: | The proposed strategy improves the performance of language models by combining heuristics with rational strategies. |