Papers by Yue Guan
Muffin or Chihuahua? Challenging Multimodal Large Language Models with Multipanel VQA (2024.acl-long)
Copied to clipboard
Yue Fan, Jing Gu, Kaiwen Zhou, Qianqi Yan, Shan Jiang, Ching-Chen Kuo, Yang Zhao, Xinze Guan, Xin Wang
| Challenge: | Multipanel images are a common form of visual representations, and humans can achieve approximately 99% accuracy on these questions. |
| Approach: | They propose a benchmark that tests multipanel visual reasoning models with 6,600 triplets of questions, answers, and multipanel images. |
| Outcome: | The proposed benchmark features 6,600 triplets of questions, answers, and multipanel images that challenge state-of-the-art Multimodal Large Language Models (MLLMs) human users can attain approximately 99% accuracy on these questions, compared with previous benchmarks. |
Read Anywhere Pointed: Layout-aware GUI Screen Reading with Tree-of-Lens Grounding (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for GUI understanding ignore a key GUI-referring task: screen reading based on user-indicated points. |
| Approach: | They propose a Tree-of-Lens agent that constructs a Hierarchical Layout Tree based on user input points and a GUI screenshot. |
| Outcome: | The proposed agent can interpret the Screen Point-and-Read task on mobile, web, and operating systems. |
Boosting Data Utilization for Multilingual Dense Retrieval (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies focus on fine-tuning multilingual dense retrieval models, but data scarcity for low-resource languages makes it difficult to align representations in a shared vector space. |
| Approach: | They propose to obtain high-quality hard negative samples and effective mini-batch data to boost data utilization for multilingual dense retrieval by obtaining high-quality negative samples. |
| Outcome: | The proposed method outperforms existing baselines on a multilingual retrieval benchmark, MIRACL, with 16 languages. |
Multimodal Inconsistency Reasoning (MMIR): A New Benchmark for Multimodal Reasoning Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing Multimodal Large Language Models (MLLMs) are predominantly trained on consistent visual-textual inputs, leaving open the question of whether they can handle semantic mismatches in layout-rich content. |
| Approach: | They propose to use multimodal inconsistency reasoning to assess MLLMs' ability to reason about semantic mismatches in webpages, presentation slides, and posters. |
| Outcome: | The proposed model outperforms open-source models in detecting inconsistencies in webpages, presentation slides, and posters while remaining vulnerable to inconsistent errors. |
Transkimmer: Transformer Learns to Layer-wise Skim (2022.acl-long)
Copied to clipboard
| Challenge: | Prior work has proposed to augment Transformer model with the capability of skimming tokens to improve its computational efficiency. |
| Approach: | They propose to add a parameterized predictor before each layer that learns to make the skimming decision. |
| Outcome: | The proposed model achieves 10.97x speedup on GLUE benchmark compared with BERT-base baseline with less than 1% accuracy degradation. |
H-LegalKI: A Hierarchical Legal Knowledge Integration Framework for Legal Community Question Answering (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Legal question answering (LQA) aims to bridge the gap between limited availability of legal professionals and the extensive volume of legal issues. |
| Approach: | They propose a legal knowledge retriever and a hierarchical legal knowledge integration framework to address multiple user-specific circumstances. |
| Outcome: | The proposed framework outperforms baselines on the legal community question-answering dataset. |
SC2: Towards Enhancing Content Preservation and Style Consistency in Long Text Style Transfer (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for short TST are difficult to implement and can cause content degradation. |
| Approach: | They propose a method to vary the style polarity of text while preserving semantic content. |
| Outcome: | The proposed method improves over baselines and is highly efficient. |
How Far Does BERT Look At: Distance-based Clustering and Analysis of BERT’s Attention (2020.coling-main)
Copied to clipboard
| Challenge: | Recent work on multi-head attention mechanism shows heuristics and clues in analyzing various aspects of the mechanism. |
| Approach: | They propose to cluster attention heatmaps into significantly different patterns through unsupervised clustering on top of a set of proposed features. |
| Outcome: | The proposed features can explain and calibrate different attention heads in Transformer models. |