Papers by Yun Fu
Advancing Vision-Language Models with Adapter Ensemble Strategies (2024.findings-emnlp)
Copied to clipboard
| Challenge: | CLIP revolutes vision-language pretraining by using contrastive learning on paired web data. |
| Approach: | They propose to combine a "adapter ensemble" with traditional machine learning techniques to augment large-scale pretrained vision-language models. |
| Outcome: | The proposed model outperforms baselines and derives improvement when the number of ensemble parameters increases. |
LIFTED: Multimodal Clinical Trial Outcome Prediction via Large Language Models and Mixture-of-Experts (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Clinical trials are costly and pivotal processes that require substantial expenses . a new approach to integrate multimodal data for clinical outcome prediction is needed . |
| Approach: | a proposed framework transforms modality-specific data into natural language descriptions . a sparse Mixture-of-Experts mechanism then identifies shared patterns across modalities . |
| Outcome: | a proposed framework outperforms baseline methods in predicting clinical trial outcomes . it transforms modality-specific data into natural language descriptions, encoded via unified encoders . |
ACBQ: Adaptive Cross-Block Quantization of Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for post-training quantization struggle to support weight–activation joint quantization and extreme low-bit weight quantization. |
| Approach: | They propose a framework that addresses weight–activation joint quantization and extreme weight quantization. |
| Outcome: | The proposed framework achieves superior performance under both W4A4 and highly aggressive W2 settings while incurring negligible additional computational overhead. |
Representation Potentials of Foundation Models for Multimodal Alignment: A Survey (2025.emnlp-main)
Copied to clipboard
| Challenge: | foundation models learn highly transferable representations through large-scale pretraining on diverse data. |
| Approach: | They examine the representation potentials of foundation models by examining their latent capacity to capture task-specific information within a single modality while providing a transferable basis for alignment and unification across modalities. |
| Outcome: | The foundation models exhibit remarkable similarities across architectures and modalities, the authors show . the models can capture task-specific information within a single modality while providing a transferable basis for alignment and unification across modality. |
Towards Unified Multimodal Large Language Models: A survey (2026.findings-acl)
Copied to clipboard
| Challenge: | unified multimodal large language models (MLLMs) are emerging but lack a systematic framework to connect them and situate current trends within a broader landscape. |
| Approach: | They present a systematic review of unified Multimodal Large Language Models . they outline the foundational concepts and prerequisites for understanding them . |
| Outcome: | The present review provides a systematic and systematic overview of unified MLLMs . it discusses persistent challenges and identify promising directions for future research . |
Revealing the Seen, Imagining the Beyond: A Survey of Image-Grounded Chain-of-Thought Reasoning in Multimodal LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in Multimodal Large Language Models (MLLMs) have shifted visual reasoning from tool-calling to end-to-end perceptionreasoning. |
| Approach: | They synthesize the emerging paradigm of Image-Grounded Chain-of-Thought (IG-CoT) they propose a method-centric taxonomy covering prompting, supervised fine-tuning, and reinforcement learning . |
| Outcome: | The proposed model is based on a method-centric taxonomy and benchmarks. |
From Words to Pixels: A Comprehensive Survey on Large Language Models in Visual Segmentation (2026.acl-long)
Copied to clipboard
| Challenge: | Visual segmentation with instruction has been a challenging task for many years . large language models and large multimodal models have spurred a new wave of research . |
| Approach: | They review recent works in LLM-based visual segmentation and analyze their architectural innovations, training strategies, and benchmark performance. |
| Outcome: | The present study reviews the most recent works in LLM-driven visual segmentation . it identifies key challenges and promising future directions . |
Cautious Next Token Prediction (2025.findings-acl)
Copied to clipboard
Yizhou Wang, Lingzhi Zhang, Yue Bai, Mang Tik Chiu, Zhengmian Hu, Mingyuan Zhang, Qihua Dong, Yu Yin, Sohrab Amirghodsi, Yun Fu
| Challenge: | Existing methods for decoding autoregressive models are temperature scaling and nucleus sampling to balance diversity and coherence. |
| Approach: | They propose a training-free decoding strategy that uses a model with a low perplexity score to select the trial with the lowest perplexities as the most probable and reliable path. |
| Outcome: | The proposed approach outperforms existing standard decoding strategies consistently by a clear margin. |
UnAC: Adaptive Visual Prompting with Abstraction and Stepwise Checking for Complex Multimodal Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent large multimodal models (LMMs) have demonstrated impressive capabilities in image understanding, yet they struggle to perform complex reasoning on multimodal problems. |
| Approach: | They propose a multimodal prompting method that strengthens reasoning for multimodal tasks in large multimodal models. |
| Outcome: | The proposed method improves reasoning on three public benchmarks and shows that it can be used to extract key information from images. |
Scaling Down, Serving Fast: Compressing and Deploying Efficient LLMs for Recommendation Systems (2025.emnlp-industry)
Copied to clipboard
Kayhan Behdin, Ata Fatahibaarzi, Qingquan Song, Yun Dai, Aman Gupta, Zhipeng Wang, Hejian Sang, Shao Tang, Gregory Dexter, Sirou Zhu, Siyu Zhu, Tejas Dharamsi, Vignesh Kothapalli, Zhoutong Fu, Yihan Cao, Pin-Lun Hsu, Fedor Borisyuk, Natesh S. Pillai, Luke Simon, Rahul Mazumder
| Challenge: | Large language models (LLMs) have demonstrated remarkable performance across a wide range of industrial applications. |
| Approach: | They propose two techniques for training and deploying small language models that deliver high performance for a variety of industry use cases. |
| Outcome: | The proposed techniques retain much of the quality of larger models while reducing training/serving costs and latency. |
MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application (2026.acl-long)
Copied to clipboard
Xueqing Peng, Lingfei Qian, Yan Wang, Ruoyu Xiang, Yueru He, Yang Ren, Mingyang Jiang, Vincent Jim Zhang, Yuqing Guo, Jeff Zhao, Huan He, Yi Han, Yun Feng, Yuechen Jiang, Yupeng Cao, Haohang Li, Yangyang Yu, Xiaoyu Wang, Penglei Gao, Shengyuan Lin, Keyi Wang, Shanshan Yang, Yilun Zhao, Zhiwei Liu, Peng Lu, Jerry Huang, Suyuchen Wang, Triantafillos Papadopoulos, Polydoros Giannouris, Efstathia Soufleri, Nuo Chen, Zhiyang Deng, Heming Fu, Yijia Zhao, Mingquan Lin, Meikang Qiu, Kaleb E Smith, Arman Cohan, Xiao-Yang Liu, Jimin Huang, Guojun Xiong, Alejandro Lopez-Lira, Xi Chen, Junichi Tsujii, Jian-Yun Nie, Sophia Ananiadou, Qianqian Xie
| Challenge: | Existing evaluations of LLMs in finance are text-only, monolingual, and largely saturated by current models. |
| Approach: | They propose a multilingual and multimodal benchmark for evaluating LLMs in real financial contexts. |
| Outcome: | The first expert-annotated multilingual and multimodal benchmark is released . it evaluates 21 leading LLMs and shows they perform better in multilingual settings . |
Distorted or Fabricated? A Survey on Hallucination in Video LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Despite significant advances in video-language modeling, hallucinations remain a persistent challenge in video large language models. |
| Approach: | They present a systematic taxonomy that categorizes hallucinations into two core types: dynamic distortion and content fabrication. |
| Outcome: | The proposed taxonomy categorizes hallucinations into two core types: dynamic distortion and content fabrication. |
NoisyCausal: A Benchmark for Evaluating Causal Reasoning Under Structured Noise (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks focus on clean, abstract scenarios where causal structure is simple or implicitly assumed. |
| Approach: | They propose a benchmark to evaluate causal reasoning under structured noise. |
| Outcome: | The proposed method outperforms standard prompting and reasoning baselines on NoisyCausal. |
D-CoDe: Scaling Image-Pretrained VLMs to Video via Dynamic Compression and Question Decomposition (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing training-free adaptation frameworks for video large language models are limited by the scarcity of high-quality video-text data. |
| Approach: | They propose a training-free adaptation framework that incorporates dynamic compression and question decomposition to address perception bottleneck and token overload. |
| Outcome: | The proposed framework improves video understanding across various benchmarks and on long-video benchmarks. |
Hierarchical Visual Agent: Managing Contexts in Joint Image-Text Space for Advanced Chart Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing MLLMs are strong at understanding single plots, but struggle with multi-step reasoning . Existing approaches to manage context in chart reasoning include text-based chain-of-thought prompting . |
| Approach: | They propose a hierarchical visual agent framework that iteratively constructs a working context in an image–text space. |
| Outcome: | The proposed framework improves on strong multimodal baselines. |