Papers by Atsushi Hashimoto
Visual Grounding Annotation of Recipe Flow Graph (2020.lrec-1)
Copied to clipboard
Taichi Nishimura, Suzushi Tomori, Hayato Hashimoto, Atsushi Hashimoto, Yoko Yamakata, Jun Harashima, Yoshitaka Ushiku, Shinsuke Mori
| Challenge: | Existing studies have ground visual observations with procedural texts with graphs to understand which objects are aligned with textual descriptions. |
| Approach: | They propose to provide visual grounding annotations to recipe flow graphs by adding bounding boxes to image sequences of recipes and annotating two types of event attributes with each bounding box. |
| Outcome: | The proposed dataset gives visual grounding with workflow’s contextual information between procedural text and visual observation in an indirect manner. |
Visual Recipe Flow: A Dataset for Learning Visual State Changes of Objects with Recipe Flows (2022.coling-1)
Copied to clipboard
Keisuke Shirai, Atsushi Hashimoto, Taichi Nishimura, Hirotaka Kameko, Shuhei Kurita, Yoshitaka Ushiku, Shinsuke Mori
| Challenge: | a new dataset enables us to learn a cooking action result for each object in a recipe text. |
| Approach: | They propose a multimodal dataset that enables us to learn a cooking action result for each object in a recipe text. |
| Outcome: | The proposed dataset reduces human annotation costs by allowing multimodal information retrieval. |
Removing Word-Level Spurious Alignment between Images and Pseudo-Captions in Unsupervised Image Captioning (2021.eacl-main)
Copied to clipboard
| Challenge: | Unsupervised image captioning is a challenging task that requires manual annotation. |
| Approach: | They propose a simple gating mechanism that is trained to align image features with the most reliable words in pseudo-captions. |
| Outcome: | The proposed method outperforms the previous methods without complex learning objectives. |