Papers by Yishu Miao
Contrastive Video-Language Learning with Fine-grained Frame Sampling (2022.aacl-main)
Copied to clipboard
| Challenge: | despite recent progress in video and language representation learning, the weak or sparse correspondence between the two modalities remains a bottleneck. |
| Approach: | They propose a fine-grained contrastive objective for video frame sampling to improve cross-modal correspondence. |
| Outcome: | The proposed approach achieves state-of-the-art performance on YouCookII with long videos. |
Discovering Topics in Long-tailed Corpora with Causal Intervention (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing topic models are designed and evaluated on balanced corpora, but the longtailed bias can impair the performance of topic models. |
| Approach: | They propose a causal inference framework to explain topic modeling on long-tailed corpora by applying causal intervention in training. |
| Outcome: | The proposed model can mitigate the bias effect, greatly improve topic quality and discover the hidden semantics on the tail. |
Exploiting Multimodal Reinforcement Learning for Simultaneous Machine Translation (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing studies on multimodality in simultaneous machine translation have highlighted the challenges for the agent to maintain good translation quality while learning an optimal translation path. |
| Approach: | They propose a multimodal approach to simultaneous machine translation using reinforcement learning with strategies to integrate visual and textual information in both the agent and the environment. |
| Outcome: | The proposed multimodal approach improves translation quality while keeping latency low while providing visual cues. |
Guiding Visual Question Generation (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to Visual Question Generation (VQG) are trained to mimic an arbitrary choice of concept but only one or a few are captured by the human references. |
| Approach: | They propose a variant of Visual Question Generation which conditions the question generator on categorical information based on expectations on the type of question and the objects it should explore. |
| Outcome: | The proposed model improves on the current state of the art on an answer-category augmented VQA dataset and human evaluation validates that guidance helps the generation of questions that are grammatically coherent and relevant to the given image and objects. |
A Generative Framework for Simultaneous Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches use a fixed number of source words to translate or learn dynamic policies for the number of sources by reinforcement learning. |
| Approach: | They propose a generative framework that uses a latent variable to model read or translate actions at every time step and integrates out to consider all possible translation policies. |
| Outcome: | The proposed framework achieves the best BLEU scores on benchmark datasets. |
Unleashing Spatial Reasoning in Multimodal Large Language Models via Textual Representation Guided Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing Multimodal Large Language Models struggle with 3D spatial reasoning as they fail to construct structured abstractions of the 3D environment depicted in video inputs. |
| Approach: | They propose a prompting method that induces MLLMs to generate 3D representations as reasoning traces for more accurate spatial question answering. |
| Outcome: | Extensive experiments on VSI-Bench and OST-Bech show that TRACE improves over prior prompting strategies across a diverse range of MLLM backbones. |
Short Text Topic Modeling with Topic Distribution Quantization and Negative Sampling Decoder (2020.emnlp-main)
Copied to clipboard
| Challenge: | Topic models for short texts suffer from data sparsity because of limited word co-occurrences. |
| Approach: | They propose a neural topic model with a new topic distribution quantization approach that generates peakier distributions that are more appropriate for modeling short texts. |
| Outcome: | The proposed model outperforms both strong traditional and neural baselines under extreme data sparsity scenes, producing high-quality topics. |