Papers by Zhiyu Lin
S2S-Arena: Evaluating Paralinguistic Instruction Following in Speech-to-Speech Models (2026.acl-long)
Copied to clipboard
Feng Jiang, Zhiyu Lin, Yiyang Liu, Liumeng Xue, Fan Bu, Yuhao Du, Xiangying Chen, Benyou Wang, Haizhou Li
| Challenge: | Existing benchmarks rely heavily on text-based evaluation and largely ignore paralinguistic cues such as prosody, emotion, and speaker traits. |
| Approach: | They propose a speech-native benchmark for evaluating instruction-following S2S models with explicit assessment of both semantic understanding and paralinguistic expression. |
| Outcome: | The proposed system enables more natural, robust, and human-aligned speech agents. |
KPatch: Knowledge Patch to Pre-trained Language Model for Zero-Shot Stance Detection on Social Media (2024.lrec-main)
Copied to clipboard
Shuohao Lin, Wei Chen, Yunpeng Gao, Zhishu Jiang, Mengqi Liao, Zhiyu Zhang, Shuyuan Zhao, Huaiyu Wan
| Challenge: | Existing knowledge injection methods fail to understand the semantics of tweets . |
| Approach: | They propose a method to flexibly inject knowledge into a pre-trained language model and adaptively expand tweets context. |
| Outcome: | The proposed method is based on two training stages to flexibly inject knowledge into the pre-trained language model and adaptively expand tweets context. |
Large Language Models as Zero-shot Dialogue State Tracker through Function Calling (2024.acl-long)
Copied to clipboard
Zekun Li, Zhiyu Chen, Mike Ross, Patrick Huber, Seungwhan Moon, Zhaojiang Lin, Xin Dong, Adithya Sagar, Xifeng Yan, Paul Crook
| Challenge: | Large language models (LLMs) are increasingly prevalent in conversational systems due to their advanced understanding and generative capabilities in general contexts. |
| Approach: | They propose a method for solving dialogue state tracking (DST) with large language models through function calling. |
| Outcome: | The proposed approach improves zero-shot DST, allowing adaptation to diverse domains without extensive data collection or model tuning. |
LLMs Caught in the Crossfire: Malware Requests and Jailbreak Challenges (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have a high vulnerability to jailbreak attacks that leverage crafted prompts to generate malicious outputs. |
| Approach: | They propose to use large language models to test their security against jailbreak attacks that leverage crafted prompts to generate malicious outputs. |
| Outcome: | The proposed model is based on 320 manually crafted malicious code generation requirements, covering 11 jailbreak methods and 29 code functionality categories. |
MT-Video-Bench: A Holistic Video Understanding Benchmark for Evaluating Multimodal LLMs in Multi-Turn Dialogues (2026.findings-acl)
Copied to clipboard
Yaning Pan, Qianqian Xie, Guohui Zhang, Zekun Moore Wang, Yongqian Wen, Yuanxing Zhang, Haoxuan Hu, Zhiyu Pan, Yibing Huang, Zhidong Gan, Yonghong Lin, An Ping, Shihao Li, Yanghai Wang, Tianhao Peng, Jiaheng Liu
| Challenge: | Existing evaluation benchmarks for Multimodal Large Language Models (MLLMs) focus on single-turn question answering, overlooking the complexity of multi-turn dialogues in real-world scenarios. |
| Approach: | They propose a video understanding benchmark for MLLMs in multi-turn dialogues that assesses six core competencies that focus on perceptivity and interactivity. |
| Outcome: | The MT-Video-Bench evaluates 1,000 multi-turn dialogues from diverse domains and reveals significant performance discrepancies and limitations in handling multi-turned video dialogues. |
ObfusLM: Privacy-preserving Language Model Service against Embedding Inversion Attacks (2025.acl-long)
Copied to clipboard
Yu Lin, Ruining Yang, Yunlong Mao, Qizhi Zhang, Jue Hong, Quanwei Cai, Ye Wu, Huiqi Liu, Zhiyu Chen, Bing Duan, Sheng Zhong
| Challenge: | Recent studies show that obfuscation techniques for MLaaS are susceptible to embedding inversion attacks (EIAs). |
| Approach: | They propose a model obfuscation framework that protects client inputs from embedding inversion attacks by obliviously obbing models. |
| Outcome: | The proposed framework outperforms existing works in utility by 10% with a nearly 80% resistance rate against embedding inversion attacks. |
WebUIBench: A Comprehensive Benchmark for Evaluating Multimodal Large Language Models in WebUI-to-Code (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for large language models focus on webpage generation outcomes. |
| Approach: | They propose a multi-view evaluation framework to evaluate MLLMs in four key areas: WebUI Perception, HTML Programming, WebUI-HTML Understanding, and WebUI to code. |
| Outcome: | The proposed framework evaluates MLLMs in four key areas: WebUI Perception, HTML Programming, WebUI-HTML Understanding, and WebUI to code. |
ThinkPilot: Steering Reasoning Models via Automated Think-prefixes Optimization (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large Reasoning Models (LRMs) are powerful but still suffer from inefficient and off-target reasoning. |
| Approach: | They propose a training-free framework that automatically optimizes Large Reasoning Models' reasoning by generating think-prefixes that evolve driven by a taxonomy of reasoning behaviors. |
| Outcome: | The proposed framework significantly improves accuracy-length trade-off for efficient reasoning, drastically improves safety and improves instruction following. |
A Generative Adaptive Replay Continual Learning Model for Temporal Knowledge Graph Reasoning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing Continual Learning (CL)-based Temporal Knowledge Graph Reasoning methods are incomplete and reorganize historical facts without preserving historical knowledge. |
| Approach: | They propose a method which generates and adaptively replays historical entity distributions from the whole historical context. |
| Outcome: | The proposed method outperforms baselines in reasoning and mitigating forgetting. |
Manual Evaluation Matters: Reviewing Test Protocols of Distantly Supervised Relation Extraction (2021.findings-acl)
Copied to clipboard
Tianyu Gao, Xu Han, Yuzhuo Bai, Keyue Qiu, Zhiyu Xie, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou
| Challenge: | Distantly supervised relation extraction (RE) has attracted much attention in the past few years . previous methods to evaluate models manually or directly on autolabeled data have produced inaccurate evaluations . |
| Approach: | They propose to use distant supervision to generate large-scale autolabeled data . they build manually-annotated test sets for two DS-RE datasets and evaluate models . |
| Outcome: | The proposed method produces 53% wrong labels at the entity pair level in the popular NYT10 dataset. |