Papers by Byoung-Tak Zhang
Modal-specific Pseudo Query Generation for Video Corpus Moment Retrieval (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown promising results in video corpus moment retrieval . however, they relied on the expensive query annotations for the VCMR . |
| Approach: | They propose a self-supervised learning framework to localize video corpus moment without annotations. |
| Outcome: | The proposed framework can localize the video corpus moment without any explicit annotation on TVR dataset. |
CoDraw: Collaborative Drawing as a Testbed for Grounded Goal-driven Communication (P19-1)
Copied to clipboard
Jin-Hwa Kim, Nikita Kitaev, Xinlei Chen, Marcus Rohrbach, Byoung-Tak Zhang, Yuandong Tian, Dhruv Batra, Devi Parikh
| Challenge: | a goal-driven collaborative drawing task combines language, perception, and actions in a partially observable environment . et al., 1990: 138K messages exchanged between human players. |
| Approach: | They propose a goal-driven collaborative task that combines language, perception, and action . they collect a clip art dataset and use it to build an image-drawing game between two agents . |
| Outcome: | The proposed task integrates language, perception, and action in a virtual world . it is based on a dataset of 10K dialogs and 138K messages exchanged between humans . |
Devil’s Advocate: Novel Boosting Ensemble Method from Psychological Findings for Text Classification (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing ensemble methods that combine submodels to create a composite model can improve model performance by diminishing model bias and variance. |
| Approach: | They propose a method which uses a deliberately dissenting model to force other submodels within the ensemble to better collaborate. |
| Outcome: | The proposed method shows comparable or improved performance on 5 text classification tasks when compared to conventional methods. |
Attend What You Need: Motion-Appearance Synergistic Networks for Video Question Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Recent advances in natural language processing and computer vision have made significant progress in artificial intelligence (AI). |
| Approach: | They propose Motion-Appearance Synergistic Networks which embed cross-modal features grounded on motion and appearance information and selectively utilize them depending on the question’s intentions. |
| Outcome: | The proposed network achieves state-of-the-art on the TGIF-QA and MSVD-QA datasets and qualitatively analyzes the results. |
Dual Attention Networks for Visual Reference Resolution in Visual Dialog (D19-1)
Copied to clipboard
| Challenge: | Visual dialog (VisDial) requires a dialog agent to answer a series of questions grounded in an image. |
| Approach: | They propose dual attention networks (DAN) for visual reference resolution in VisDial. |
| Outcome: | The proposed model outperforms the previous state-of-the-art model on VisDial datasets. |
Hypergraph Transformer: Weakly-Supervised Multi-hop Reasoning for Knowledge-based Visual Question Answering (2022.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge-based visual question answering tasks require weak supervision and no visual knowledge. |
| Approach: | They propose a model which encodes high-level semantics of a question and a knowledge base and learns high order associations between them. |
| Outcome: | The proposed model encodes high-level semantics of a question and a knowledge base, and learns high order associations between them. |
Confidence-guided Refinement Reasoning for Zero-shot Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing frameworks that generate single-step reasoning do not improve QA reasoning . |
| Approach: | They propose a framework that strategically constructs and refines sub-questions and their answers (sub-QAs) they argue that sub-QA does not always enhance QA reasoning . |
| Outcome: | The proposed framework can be integrated with existing QA models and benchmarks. |
Reasoning Visual Dialog with Sparse Graph Learning and Knowledge Transfer (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Visual dialog is a task of answering questions grounded in an image using dialog history as context. |
| Approach: | They propose a Sparse Graph Learning method to formulate visual dialog as a graph structure learning task. |
| Outcome: | The proposed model outperforms the state-of-the-art models on the VisDial v1.0 dataset. |