Papers by Yapeng Tian
Self-Improvement in Multimodal Large Language Models: A Survey (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Using data and data, self-improvement for Large Language Models has improved model capabilities without significantly increasing costs. |
| Approach: | This survey provides a comprehensive overview of self-improvement for Large Language Models . it includes commonly used evaluations and downstream applications . |
| Outcome: | The authors provide a comprehensive overview of self-improvement in Multimodal LLMs. |
OSCaR: Object State Captioning and State Change Representation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to extrapolate and comprehend changes in object states are limited . relying on a small set of symbolic words to represent changes has restricted expressiveness of language. |
| Approach: | They propose a dataset and benchmark to evaluate multimodal large language models . they investigate causal relations between a concrete action and the change . |
| Outcome: | The proposed method achieves near parity with GPT-4V ratings across helpfulness, accuracy, reasoning, and other key metrics. |
CLIPErase: Efficient Unlearning of Visual-Textual Associations in CLIP (2025.acl-long)
Copied to clipboard
| Challenge: | MU has gained significant attention as a means to remove the influence of specific data from a trained model without requiring full retraining. |
| Approach: | They propose a novel approach that disentangles and selectively forgets both visual and textual associations, ensuring that unlearning does not compromise model performance. |
| Outcome: | Experiments on CIFAR-100, Flickr30K, and Conceptual 12M show that CLIPErase effectively removes designated associations from multimodal samples in downstream tasks while preserving model performance on retain set. |
SaSR-Net: Source-Aware Semantic Representation Network for Enhancing Audio-Visual Question Answering (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing AVQA methods often fail to link sound-producing objects in the video with the audio-visual information. |
| Approach: | They introduce a source-aware semantic representation network for AVQA . they use source-wise learnable tokens to capture and align audio-visual elements with the question . |
| Outcome: | The proposed model outperforms state-of-the-art models on the Music-AVQA and AVQA-Yang datasets. |