Papers by Zhenhai Zhu
Beyond Instructional Videos: Probing for More Diverse Visual-Textual Grounding on YouTube (2020.emnlp-main)
Copied to clipboard
| Challenge: | a representative pretraining model is fit to a diverse YouTube8M dataset . a priori, this domain is relatively easy for instructional videos . |
| Approach: | They fit a representative pretraining model to a YouTube8M dataset and examine its success and failure cases. |
| Outcome: | The proposed model can be trained on more diverse video corpora and achieve high performance on many video understanding tasks. |
Multimodal Pretraining for Dense Video Captioning (2020.aacl-main)
Copied to clipboard
| Challenge: | a billion hours of videos are being watched on YouTube every day . videos are difficult to skim through, making it harder to quickly target the relevant part(s) of a video. |
| Approach: | They propose to use a video timeline tag dataset to generate time-stamped annotations for videos . they propose to pretrain and finetune captioning models using YouCook2 and ViTT . |
| Outcome: | The proposed model generalizes well and is robust over a wide variety of instructional videos. |
H-Transformer-1D: Fast One-Dimensional Hierarchical Attention for Sequences (2021.acl-long)
Copied to clipboard
| Challenge: | Existing hierarchical methods to compute attention are superior to sub-quadratic ones . a particular type of attention, called multi-head scaled dot-product attention, is one of the main components of the Transformer architecture . |
| Approach: | They propose a hierarchical method to compute attention in the Transformer architecture . they perform extensive experiments to show that it captures hierarchic structure in sequences . |
| Outcome: | The proposed method outperforms sub-quadratic models on the Long Range Arena benchmark by over +6 points on average. |