Papers by Jianfei Zhao
Mitigating Hallucination in Large Vision-Language Models through Aligning Attention Distribution to Information Flow (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Decode-Only models propagate information from left to right, but the model's attention still focuses on the visual representations, resulting in hallucinations. |
| Approach: | They propose to leverage the core information embedded in semantic representations to enhance the model's visual understanding by leveraging the attention distributions. |
| Outcome: | The proposed method reduces hallucinations by 80% by aligning the attention distribution with the actual information flow. |
Cross-Domain Data Augmentation with Domain-Adaptive Language Modeling for Aspect-Based Sentiment Analysis (2023.acl-long)
Copied to clipboard
| Challenge: | Cross-domain Aspect-Based Sentiment Analysis (ABSA) aims to identify aspect-sentiment pairs in sentences from a target domain. |
| Approach: | They propose a domain-adaptive language model to generate labeled data from a source domain. |
| Outcome: | The proposed approach outperforms existing methods on ABSA and Aspect Extraction tasks. |
Bridging the Temporal Gap in Multimodal LLMs: Deeply Stacking Temporal Tokens for Audio-Visual Speech Recognition (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing audio-visual speech recognition systems suffer from a temporal gap . visual speech patterns captured from lip movements provide complementary information that remains inherently robust to acoustic noise. |
| Approach: | They propose a framework that deeply stacks temporal tokens across both encoding and decoding stages to bridge this temporal gap. |
| Outcome: | The proposed framework outperforms existing supervised, self-supervised, and LLM-based methods by 6.1% on LRS2 and 7.8% on LLS3. |
Learning to Look at the Other Side: A Semantic Probing Study of Word Embeddings in LLMs with Enabled Bidirectional Attention (2025.acl-long)
Copied to clipboard
| Challenge: | Autoregressive Large Language Models (LLMs) demonstrate exceptional performance in language understanding and generation tasks, but their application in text embedding tasks has been relatively slow due to the constraints of the unidirectional attention mechanism. |
| Approach: | They propose to enable bidirectional attention in LLMs to improve their ability to represent subsequent context but weaken their utilization of preceding context. |
| Outcome: | The proposed model improves the LLMs’ ability to represent subsequent context but weakens their utilization of preceding context, while contrastive learning training can help to maintain both abilities. |