Papers by Yoshitaka Ushiku
Visual Grounding Annotation of Recipe Flow Graph (2020.lrec-1)
Copied to clipboard
Taichi Nishimura, Suzushi Tomori, Hayato Hashimoto, Atsushi Hashimoto, Yoko Yamakata, Jun Harashima, Yoshitaka Ushiku, Shinsuke Mori
| Challenge: | Existing studies have ground visual observations with procedural texts with graphs to understand which objects are aligned with textual descriptions. |
| Approach: | They propose to provide visual grounding annotations to recipe flow graphs by adding bounding boxes to image sequences of recipes and annotating two types of event attributes with each bounding box. |
| Outcome: | The proposed dataset gives visual grounding with workflow’s contextual information between procedural text and visual observation in an indirect manner. |
Visual Recipe Flow: A Dataset for Learning Visual State Changes of Objects with Recipe Flows (2022.coling-1)
Copied to clipboard
Keisuke Shirai, Atsushi Hashimoto, Taichi Nishimura, Hirotaka Kameko, Shuhei Kurita, Yoshitaka Ushiku, Shinsuke Mori
| Challenge: | a new dataset enables us to learn a cooking action result for each object in a recipe text. |
| Approach: | They propose a multimodal dataset that enables us to learn a cooking action result for each object in a recipe text. |
| Outcome: | The proposed dataset reduces human annotation costs by allowing multimodal information retrieval. |
Where is the answer? An empirical study of positional bias for parametric knowledge extraction in language model (2025.naacl-long)
Copied to clipboard
| Challenge: | Language model (LM) stores diverse factual knowledge in their parameters, which is learned during self-supervised training on unlabeled documents. |
| Approach: | They investigate the issue of "perplexity curse" in the continued training of language model (LM) they find that all studied LMs suffer from positional bias in the training document . |
| Outcome: | The proposed model is able to extract information from multiple questions with diverse queries. |
Removing Word-Level Spurious Alignment between Images and Pseudo-Captions in Unsupervised Image Captioning (2021.eacl-main)
Copied to clipboard
| Challenge: | Unsupervised image captioning is a challenging task that requires manual annotation. |
| Approach: | They propose a simple gating mechanism that is trained to align image features with the most reliable words in pseudo-captions. |
| Outcome: | The proposed method outperforms the previous methods without complex learning objectives. |