Papers by Qingming Huang
Think Beyond Words: Exploring Context-Relevant Visual Commonsense for Diverse Dialogue Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to generate intelligent open-domain dialogue agents only consider auxiliary commonsense stored in pure text, ignoring grounding information from the external visual world. |
| Approach: | They propose a VIsual Commonsense enhanced dialogue generaTOR that exploits auxiliary commonsense from images related to context to generate coherent and informative responses. |
| Outcome: | The proposed method outperforms the latest competitive methods in terms of coherence and diversity on two public datasets. |
The Devil is in the Distributions: Explicit Modeling of Scene Content is Key in Zero-Shot Video Captioning (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for zero-shot video captioning focus on one key aspect of the scene and ignore the rest of the visual input. |
| Approach: | They propose a novel textual prompting strategy for zero-shot video captioning that uses a category-aware retrieval mechanism to promote prompt diversity while ensuring visual relevance. |
| Outcome: | The proposed method outperforms existing methods on in-domain and cross-domain settings. |
Change Entity-guided Heterogeneous Representation Disentangling for Change Captioning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to describe differences between two images are highly challenging due to distractors such as illumination and viewpoint changes. |
| Approach: | They propose a change-entity-guided disentanglement network that explicitly learns difference representations while mitigating the impact of distractors. |
| Outcome: | The proposed method outperforms existing methods on CLEVR-Change, CLE VR-DC and Spot-the-Diff datasets and achieves state-of-the art performance. |
Context-aware Difference Distilling for Multi-change Captioning (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for multi-change captioning are difficult because it requires a higher level of cognition to reason an arbitrary number of changes. |
| Approach: | They propose a context-aware difference distilling network to capture all genuine changes for yielding sentences. |
| Outcome: | The proposed network captures all genuine changes for yielding sentences on three public datasets. |
StyleDubber: Towards Multi-Scale Style Learning for Movie Dubbing (2024.findings-acl)
Copied to clipboard
Gaoxiang Cong, Yuankai Qi, Liang Li, Amin Beheshti, Zhedong Zhang, Anton Hengel, Ming-Hsuan Yang, Chenggang Yan, Qingming Huang
| Challenge: | Existing methods for movie dubbing break phonemes in scripts, resulting in incomplete phoneme pronunciation and poor identity stability. |
| Approach: | They propose a method that switches dubbing learning from frame level to phoneme level . it uses a multimodal style adaptor to learn pronunciation style from audio . |
| Outcome: | The proposed method improves on two benchmarks, V2C and Grid, and is available on github. |