Papers by Ziyu Chen
UniSonate: A Unified Model for Speech, Music, and Sound Effect Generation with Text Instructions (2026.acl-long)
Copied to clipboard
Chunyu Qiang, Xiaopeng Wang, Kang Yin, Yuzhe Liang, Yuxin Guo, Teng Ma, Ziyu Zhang, Tianrui Wang, Cheng Gong, Yushen Chen, Ruibo Fu, Longbiao Wang, Jianwu Dang
| Challenge: | Generative audio modeling has been fragmented into specialized tasks such as text-to-speech (TTS), text- to-music (TTM), and text-ta (TTA) specialized models require reference audio for timbre cloning and strict phoneme alignment, whereas TTA models generate unstructured textures from open-ended captions. |
| Approach: | They propose a unified flow-matching framework capable of synthesizing speech, music, sound effects . they propose 'token injection mechanism' that projects unstructured environmental sounds into structured temporal latent space . |
| Outcome: | The proposed framework achieves state-of-the-art performance in instruction-based TTS and TTM while maintaining competitive fidelity in TTA. |
Towards Order Fairness: Mitigating LLMs Order Sensitivity through Dual Group Advantage Optimization (2026.acl-long)
Copied to clipboard
| Challenge: | Recent studies attempt to obtain optimal or suboptimal arrangements based on statistical results or using dataset-based search, but these methods increase inference overhead while leaving the model’s inherent order bias unresolved. |
| Approach: | They propose Dual Group Advantage Optimization (DGAO) which aims to improve model accuracy and order stability simultaneously. |
| Outcome: | The proposed method improves model accuracy and order stability while penalizing order-sensitive or incorrect responses. |
InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Model (2025.findings-acl)
Copied to clipboard
Yuhang Zang, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Ziyu Liu, Shengyuan Ding, Shenxi Wu, Yubo Ma, Haodong Duan, Wenwei Zhang, Kai Chen, Dahua Lin, Jiaqi Wang
| Challenge: | Despite the promising performance of Large Vision Language Models, they sometimes generate incorrect outputs. |
| Approach: | They propose a multi-modal reward model that aligns LVLMs with human preferences. |
| Outcome: | The proposed model achieves excellent results on the latest multi-modal reward model benchmark and shows competitive performance on text-only reward model. |
SciMDR: Advancing Scientific Multimodal Document Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Current models struggle to provide reliable assistance in real-world scientific workflows because evidence is distributed across long, multimodal documents. |
| Approach: | They propose a framework for QA Synthesis and document-scale regrounding that generates faithful, isolated QA pairs and reasoning on focused segments. |
| Outcome: | The proposed framework achieves significant improvements across multiple QA benchmarks, particularly in tasks requiring complex document-level reasoning. |
Thinking with Map: Reinforced Parallel Map-Augmented Agent for Geolocalization (2026.findings-acl)
Copied to clipboard
Yuxiang Ji, Yong Wang, Ziyu Ma, Yiming Hu, Hailang Huang, Xuecai Hu, Guanhua Chen, Liaoni Wu, Xiangxiang Chu
| Challenge: | Existing large vision-language model (LVLM) approaches overlook a common strategy used by humans — using maps. |
| Approach: | They propose a method to equip a vision-language model with the ability to think with maps and optimize it using agentic reinforcement learning and parallel test-time scaling. |
| Outcome: | The proposed method outperforms open- and closed-source models on most metrics. |
Dashboard2Code: Evaluating Multimodal Models on Reconstructing Interactive Dashboards (2026.acl-long)
Copied to clipboard
Tianhao Niu, Ziyu Han, Qiguang Chen, Shiqi Zhou, Baocai Shan, Hengjie Fang, Qingfu Zhu, Wanxiang Che
| Challenge: | Existing efforts to generate static visualizations focus on static charts and interactive dashboards. |
| Approach: | They propose a dashboard2code task that requires a model to explore an interactive dashboard, acquire feedback from its own interactions and generate code that reproduces the target dashboard. |
| Outcome: | The proposed task is based on 180 carefully designed and manually verified dashboard–code pairs spanning three difficulty levels and covering eight common real-world interaction patterns. |
SciVerse: Unveiling the Knowledge Comprehension and Visual Reasoning of LMMs on Multi-modal Scientific Problems (2025.findings-acl)
Copied to clipboard
| Challenge: | SciVerse is a multi-modal scientific evaluation benchmark to assess large multi-models . it examines the scientific knowledge comprehension, multi-mod content interpretation and Chain-of-Thought reasoning . authors examine the scientific proficiency of LMMs in scientific domains based on their work . |
| Approach: | They propose a multi-modal scientific evaluation benchmark to thoroughly assess Large Multi-modal Models across 5,735 test instances in five different versions. |
| Outcome: | The proposed evaluation reveals critical limitations in LMMs' scientific proficiency and provides new insights into future developments. |
MoVa: Towards Generalizable Classification of Human Morals and Values (2025.emnlp-main)
Copied to clipboard
Ziyu Chen, Junfei Sun, Chenxi Li, Tuan Dung Nguyen, Jing Yao, Xiaoyuan Yi, Xing Xie, Chenhao Tan, Lexing Xie
| Challenge: | Identifying human morals and values embedded in language is essential to empirical studies of communication. |
| Approach: | They propose a framework for generalizable classification of human morals and values . they recommend a classification strategy that scores all related concepts simultaneously . |
| Outcome: | The proposed method outperforms fine-tuned models across domains and frameworks. |
Cross-lingual Multimodal Sentiment Analysis for Low-Resource Languages via Language Family Disentanglement and Rethinking Transfer (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing multimodal sentiment analysis methods are limited to textual data and cannot handle multimodal scenarios. |
| Approach: | They propose a transfer learning framework that allows cross-lingual and cross-modal alignments and a language family disentanglement module that enhances the sharing of language universals within families. |
| Outcome: | The proposed method is superior to existing methods and can handle low-resource languages. |
LaoPLM: Pre-trained Language Models for Lao (2022.lrec-1)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) can capture different levels of concepts in context . previous work on Lao has been hampered by the lack of annotated datasets . |
| Approach: | They construct a text classification dataset to alleviate the resource-scarce situation of Lao . they evaluate them on two downstream tasks: part-of-speech tagging and text classification . |
| Outcome: | The proposed model can capture different levels of concepts in context and generate universal language representations. |
OS Agents: A Survey on MLLM-based Agents for Computer, Phone and Browser Use (2025.acl-long)
Copied to clipboard
Xueyu Hu, Tao Xiong, Biao Yi, Zishu Wei, Ruixuan Xiao, Yurun Chen, Jiasheng Ye, Meiling Tao, Xiangxin Zhou, Ziyu Zhao, Yuhuai Li, Shengze Xu, Shenzhi Wang, Xinchen Xu, Shuofei Qiao, Zhaokai Wang, Kun Kuang, Tieyong Zeng, Liang Wang, Jiwei Li, Yuchen Eleanor Jiang, Wangchunshu Zhou, Guoyin Wang, Keting Yin, Zhou Zhao, Hongxia Yang, Fan Wu, Shengyu Zhang, Fei Wu
| Challenge: | a new generation of (M)LLMs is enabling the creation of superintelligent AI assistants . OS Agents can complete tasks autonomously and have the potential to significantly enhance the lives of billions of users worldwide. |
| Approach: | They propose to build OS Agents that operate within operating systems' GUIs and GUIs . they examine evaluation metrics and benchmarks to identify promising directions . |
| Outcome: | The proposed agents are based on operating systems (OS) and operating systems frameworks. |