Papers by Fengzong Lian
LightVLP: A Lightweight Vision-Language Pre-training via Gated Interactive Masked AutoEncoders (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing vision-language pre-training models use multi-modal encoders to encode image and text, causing noisy training corpora. |
| Approach: | They propose a vision-language pre-training framework with two autoencoders for efficient training . they propose masked tokens and a gated interaction mechanism to cope with noise . |
| Outcome: | The proposed model achieves 2.2% R@1 gains on COCO Text Retrieval and 1.1% on refCOCO+ on six datasets. |
An Anchor-based Relative Position Embedding Method for Cross-Modal Tasks (2022.emnlp-main)
Copied to clipboard
| Challenge: | Position Embedding (PE) is essential for transformer to capture the sequence ordering of input tokens. |
| Approach: | They propose a unified position embedding method that bridges the semantic gap between modalities and embeds the anchor-based distance to guide computation of cross-attention. |
| Outcome: | The proposed method obtains new SOTA results on a wide range of benchmarks. |