Papers by Kaiwen Yan
Noise-robust Cross-modal Interactive Learning with Text2Image Mask for Multi-modal Neural Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies on multi-modal neural machine translation focus on visual information, but text and image may not match exactly, and visual noise is often ignored. |
| Approach: | They propose a noise-robust multi-modal interactive fusion approach with cross-modal relation-aware mask mechanism for MNMT. |
| Outcome: | The proposed model achieves state-of-the-art scores in all En-De, En-Fr and En-Cs translation tasks. |
SafeSteer: A Decoding-level Defense Mechanism for Multimodal Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing defense methods rely on fine-tuning or inefficient post-hoc interventions, limiting their ability to address novel attacks. |
| Approach: | They propose a decoding-level defense mechanism that employs a lightweight discriminator to iteratively steer the decoding process toward safety. |
| Outcome: | The proposed method improves safety performance by up to 33.40% without fine-tuning on multiple MLLMs. |
Muffin or Chihuahua? Challenging Multimodal Large Language Models with Multipanel VQA (2024.acl-long)
Copied to clipboard
Yue Fan, Jing Gu, Kaiwen Zhou, Qianqi Yan, Shan Jiang, Ching-Chen Kuo, Yang Zhao, Xinze Guan, Xin Wang
| Challenge: | Multipanel images are a common form of visual representations, and humans can achieve approximately 99% accuracy on these questions. |
| Approach: | They propose a benchmark that tests multipanel visual reasoning models with 6,600 triplets of questions, answers, and multipanel images. |
| Outcome: | The proposed benchmark features 6,600 triplets of questions, answers, and multipanel images that challenge state-of-the-art Multimodal Large Language Models (MLLMs) human users can attain approximately 99% accuracy on these questions, compared with previous benchmarks. |
PaCoRe: Learning to Scale Test-Time Compute with Parallel Coordinated Reasoning (2026.acl-long)
Copied to clipboard
Jingcheng Hu, Yinmin Zhang, Shijie Shang, Xiaobo Yang, Yue Peng, Zhewei Huang, Hebin Zhou, Xin Wu, Jie Cheng, Fanqi Wan, Xiangwen Kong, Chengyuan Yao, Kaiwen Yan, Ailin Huang, Hongyu Zhou, Qi Han, Zheng Ge, Xiangyu Zhang, Heung-Yeung Shum
| Challenge: | Parallel Coordinated Reasoning (PaCoRe) overcomes a central limitation of contemporary language models: their inability to scale test-time compute (TTC) far beyond sequential reasoning under a fixed context window. |
| Approach: | They propose a training-and-inference framework to overcome a central limitation of language models: their inability to scale test-time compute (TTC) under a fixed context window. |
| Outcome: | The proposed model scales to multi-million-token effective TTC without exceeding context limits. |
RSA-Bench: Benchmarking Audio Large Models in Real-World Acoustic Scenarios (2026.findings-acl)
Copied to clipboard
Yibo Zhang, Kaiwen Luo, Liang Lin, Shilinlu Yan, Jin Wang, Yaoqi Guo, Yitian Chen, Yalan Qin, Zhenhong Zhou, Kun Wang, Li Sun
| Challenge: | Existing evaluations rely on synthetic Gaussian noise or simplistic single-source interference, failing to capture the intricate, multi-layered acoustic dynamics that characterize authentic physical environments. |
| Approach: | They propose a robustness benchmark to stress-test Audio Large Models (ALLMs) using high-fidelity auditory scene simulations. |
| Outcome: | The proposed model performs well on a wide range of tasks, including automatic speech recognition, speech translation, and audio-based reasoning. |
Me-Agent: A Personalized Mobile Agent with Two-Level User Habit Learning for Enhanced Interaction (2026.findings-acl)
Copied to clipboard
Shuoxin Wang, Chang Liu, Gowen Loo, Lifan Zheng, Kaiwen Wei, Huanqian Yan, Xinyi Zeng, Jingyuan Zhang, Yu Tian
| Challenge: | Existing Large Language Model (LLM)-based mobile agents follow explicit user instructions without personalized needs. |
| Approach: | They propose a user preference learning strategy enhanced with a Personal Reward Model to improve personalization performance. |
| Outcome: | The proposed agent achieves state-of-the-art performance while maintaining competitive instruction execution performance. |
CodeIF: Benchmarking the Instruction-Following Capabilities of Large Language Models for Code Generation (2025.acl-industry)
Copied to clipboard
| Challenge: | CodeIF assesses the ability of large language models to adhere to task-oriented instructions in code generation tasks. |
| Approach: | They introduce a benchmark designed to assess LLMs' ability to adhere to task-oriented instructions within diverse code generation scenarios. |
| Outcome: | The proposed benchmark assesses LLMs' ability to adhere to task-oriented instructions in code generation tasks across a wide range of complexity levels and programming domains. |