Papers by Hikaru Tomonari
JFLD: A Japanese Benchmark for Deductive Reasoning Based on Formal Logic (2024.lrec-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have proficiently solved a broad range of tasks with their rich knowledge but struggle with logical reasoning. |
| Approach: | They propose a deductive reasoning benchmark for Japanese that assesses logical reasoning abilities isolated from knowledge and various reasoning rules. |
| Outcome: | The proposed benchmarks assess whether LLMs can generate logical steps to (dis)prove a given hypothesis based on a set of facts. |
Is Micro Domain-Adaptive Pre-Training Effective for Real-World Operations? Multi-Step Evaluation Reveals Potential and Bottlenecks (2026.eacl-industry)
Copied to clipboard
Masaya Tsunokake, Yuta Koreeda, Terufumi Morishita, Koichi Nagatsuka, Hikaru Tomonari, Yasuhiro Sogawa
| Challenge: | Domain-adaptive pre-training (DAPT) is one approach for enabling LLMs to handle unseen knowledge. |
| Approach: | They propose to disentangle the answering process into three subtasks and evaluate the performance of each subtask. |
| Outcome: | The proposed model resolves the elicitation task that the base model struggled with but does not resolve other subtasks. |
Robustness Evaluation of Text Classification Models Using Mathematical Optimization and Its Application to Adversarial Training (2022.findings-aacl)
Copied to clipboard
| Challenge: | Neural networks are vulnerable to adversarial examples due to slightly perturbed input data. |
| Approach: | They propose a method that evaluates the robustness of text classification models by an optimization problem that identifies a minimum synonym swap that changes the classification result. |
| Outcome: | The proposed method achieves high scores in human evaluations of grammatical correctness and semantic similarity for an IMDb dataset and implements adversarial training with the IMD and SST2 datasets. |