Papers by Tamara Berg
mTVR: Multilingual Moment Retrieval in Videos (2021.acl-short)
Copied to clipboard
| Challenge: | mTVR is a multilingual video moment retrieval dataset with 218K queries in English and Chinese . Various datasets have been proposed or adapted for the task, but they are all created for a single language (English). |
| Approach: | They propose a multilingual video moment retrieval dataset with 218K queries from 21.8K TV show video clips. |
| Outcome: | The proposed model outperforms strong monolingual baselines while using fewer parameters. |
Revealing Single Frame Bias for Video-and-Language Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for video-and-language learning use multiple frames as inputs. |
| Approach: | They propose to use single-frame models for video-and-language learning to investigate temporality in video- and language tasks. |
| Outcome: | The proposed model does not take into account temporal information on video-and-language tasks. |
MART: Memory-Augmented Recurrent Transformer for Coherent Video Paragraph Captioning (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for generating paragraph descriptions for videos require a coherent paragraph and a higher level of coherence. |
| Approach: | They propose a new method that generates a summarized memory state from video segments and sentence history to help better predict the next sentence. |
| Outcome: | The proposed method generates more coherent and less repetitive paragraph captions while maintaining relevance to the input video events. |
What is More Likely to Happen Next? Video-and-Language Future Event Prediction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models cannot make multimodal commonsense predictions of future events based on video and dialogue . |
| Approach: | They propose a task to predict which event is more likely to happen in a video clip . they use a dataset with 28,726 future event prediction examples from 10,234 videos . |
| Outcome: | The proposed model provides a good starting point but leaves room for future work. |
TVQA+: Spatio-Temporal Grounding for Video Question Answering (2020.acl-main)
Copied to clipboard
| Challenge: | Existing video QA datasets only contain QA pairs without labels for key clips or regions needed to answer the question. |
| Approach: | They propose a framework that grounds evidence in both spatial and temporal domains to answer questions about videos using bounding boxes. |
| Outcome: | The proposed framework can produce interpretable spatio-temporal attention visualizations. |
TVQA: Localized, Compositional Video Question Answering (D18-1)
Copied to clipboard
| Challenge: | Recent studies have focused on image-based question-answering (QA) tasks, but little has been done on video-based QA. |
| Approach: | They present a large-scale video QA dataset based on 6 popular TV shows . they provide analysis of the new dataset and trainable neural network framework . |
| Outcome: | The proposed dataset includes 152,545 QA pairs from 21,793 clips spanning over 460 hours of video. |