Papers by Ana Brassard
Context Limitations Make Neural Language Models More Human-Like (2022.emnlp-main)
Copied to clipboard
| Challenge: | Language models (LMs) have been used in cognitive modeling and engineering studies to simulate human cognitive load during reading. |
| Approach: | They propose to constrain LMs' context access to improve their simulation of human reading behavior by incorporating syntactic biases into their context access. |
| Outcome: | The proposed model improves the simulation of human reading behavior by incorporating syntactic biases into their context access. |
LLMs Faithfully and Iteratively Compute Answers During CoT: A Systematic Analysis With Multi-step Arithmetics (2026.findings-eacl)
Copied to clipboard
Keito Kudo, Yoichi Aoki, Tatsuki Kuribayashi, Shusaku Sone, Masaya Taniguchi, Ana Brassard, Keisuke Sakaguchi, Kentaro Inui
| Challenge: | Specifically, we examine when the LLMs’ answer is (pre)determined, especially before the CoT begins or after, and how strongly the information from CoT specifically has a causal effect on the final answer. |
| Approach: | They examine when the LLMs’ answer is (pre)determined, especially before the CoT begins or after, and how strongly the information from CoT specifically has a causal effect on the final answer. |
| Outcome: | The proposed model can generate reasoning chains while generating the reasoning chain on the fly. |
Prompting for explanations improves Adversarial NLI. Is this true? {Yes} it is {true} because {it weakens superficial cues} (2023.findings-eacl)
Copied to clipboard
| Challenge: | Explanation prompts are used to generate an explanation for a given input . they are also used to improve model performance on adversarial benchmarks . |
| Approach: | They propose to use explanation prompts to generate an explanation that supports a label . they argue that prompting for explanations weakens superficial cues . |
| Outcome: | The proposed explanation prompts outperform label-only prompts on adversarial benchmarks. |
COPA-SSE: Semi-structured Explanations for Commonsense Reasoning (2022.lrec-1)
Copied to clipboard
| Challenge: | Semi-structured explanations for Choice of Plausible Alternatives (COPA-SSE) are a crowdsourced dataset of 9,747 common sense explanations . |
| Approach: | They propose a semi-structured approach to explain Choice of Plausible Alternatives questions using a crowdsourced dataset of 9,747 common sense explanations with ConceptNet relations but freely written concepts. |
| Outcome: | The proposed explanations are geared towards commonsense reasoners operating on knowledge graphs and serve as a starting point for improving such systems. |
Empirical Investigation of Neural Symbolic Reasoning Strategies (2023.findings-eacl)
Copied to clipboard
Yoichi Aoki, Keito Kudo, Tatsuki Kuribayashi, Ana Brassard, Masashi Yoshikawa, Keisuke Sakaguchi, Kentaro Inui
| Challenge: | Neural reasoning accuracy improves when generating intermediate reasoning steps. |
| Approach: | They decompose the reasoning strategy w.r.t. step granularity and chaining strategy. |
| Outcome: | The proposed reasoning strategy significantly affects performance in a symbolic reasoning dataset. |
Quantifying the Influence of Evaluation Aspects on Long-Form Response Assessment (2025.coling-main)
Copied to clipboard
| Challenge: | Evaluating the outputs of large language models (LLMs) on long-form generative tasks remains challenging. |
| Approach: | They propose to compute an overall quality score as a weighted average of factuality, informative-ness, and formality as compared to previous metrics. |
| Outcome: | The proposed method achieves stronger correlations with human judgments compared to previous metrics. |
Do Deep Neural Networks Capture Compositionality in Arithmetic Reasoning? (2023.eacl-main)
Copied to clipboard
Keito Kudo, Yoichi Aoki, Tatsuki Kuribayashi, Ana Brassard, Masashi Yoshikawa, Keisuke Sakaguchi, Kentaro Inui
| Challenge: | Using a pre-trained dataset, we examine how well recent neural models capture compositionality in symbolic reasoning tasks. |
| Approach: | They propose a skill tree on compositionality that defines hierarchical levels of complexity along with three compositionality dimensions: systematicity, productivity, and substitutivity. |
| Outcome: | The proposed model struggled most with systematicity, performing poorly even with relatively simple compositions. |
Evaluating Model Alignment with Human Perception: A Study on Shitsukan in LLMs and LVLMs (2025.coling-main)
Copied to clipboard
| Challenge: | This work examines the alignment of large language models and large vision-language models with human perception. |
| Approach: | They use a dataset of *shitsukan* terms elicited from individuals in response to object images to evaluate their understanding of the Japanese concept of shitukan. |
| Outcome: | The proposed models demonstrated mixed accuracy across benchmark tasks, with limited overlap between model- and human-generated terms. |
Learning to Learn to be Right for the Right Reasons (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work shows that models trained on held-out data perform poorly on hard instances . previous methods have resorted to manual methods of encouraging models not to overfit to superficial cues . |
| Approach: | They propose to explicitly learn a model that does well on both easy and hard tests . they use Choice of Plausible Alternatives and Commonsense Explanation to evaluate the model . |
| Outcome: | The proposed model performs well on easy and hard tests with superficial cues but performs poorly on hard ones without superficial cuings. |
AgentCoMa: A Compositional Benchmark Mixing Commonsense and Mathematical Reasoning in Real-World Scenarios (2026.acl-long)
Copied to clipboard
| Challenge: | brittleness of Large Language Models in reasoningintensive tasks is a problem . current compositional benchmarks focus on *either* commonsense or math reasoning . |
| Approach: | They propose a "Co**mmonsense and Ma**th" benchmark where each compositional task requires a commonsense reasoning step *and* a math reasoning step. |
| Outcome: | The proposed benchmarks show that LLMs can solve both steps in isolation, but their accuracy drops by nearly 30% when the two steps are combined. |