Papers by Zangwei Zheng
How Does the Textual Information Affect the Retrieval of Multimodal In-Context Learning? (2024.emnlp-main)
Copied to clipboard
| Challenge: | MLLMs have significant capabilities for multimodal in-context learning, but their effectiveness hinges on the appropriate selection of in-constext examples. |
| Approach: | They propose a supervised MLLM prompt retriever that leverages a trained retriever based on MLML's confidence to select examples, which enhances multimodal in-context learning efficiency. |
| Outcome: | The proposed method is validated through extensive testing across three different tasks and demonstrates its effectiveness. |
CAME: Confidence-guided Adaptive Memory Efficient Optimization (2023.acl-long)
Copied to clipboard
| Challenge: | Existing memory-efficient methods require second-moment estimates of the per-parameter gradients to maintain their performance. |
| Approach: | They propose to use memory-efficient optimizers to reduce memory usage by preserving second-moment estimates of gradients. |
| Outcome: | The proposed method achieves fast convergence and lower memory usage across training tasks. |