Papers by Yiyang Liu
S2S-Arena: Evaluating Paralinguistic Instruction Following in Speech-to-Speech Models (2026.acl-long)
Copied to clipboard
Feng Jiang, Zhiyu Lin, Yiyang Liu, Liumeng Xue, Fan Bu, Yuhao Du, Xiangying Chen, Benyou Wang, Haizhou Li
| Challenge: | Existing benchmarks rely heavily on text-based evaluation and largely ignore paralinguistic cues such as prosody, emotion, and speaker traits. |
| Approach: | They propose a speech-native benchmark for evaluating instruction-following S2S models with explicit assessment of both semantic understanding and paralinguistic expression. |
| Outcome: | The proposed system enables more natural, robust, and human-aligned speech agents. |
Model Composition for Multimodal Large Language Models (2024.acl-long)
Copied to clipboard
Chi Chen, Yiyang Du, Zheng Fang, Ziyue Wang, Fuwen Luo, Peng Li, Ming Yan, Ji Zhang, Fei Huang, Maosong Sun, Yang Liu
| Challenge: | Existing methods for creating versatile MLLMs rely on joint training with paired instruction data, which is resource-intensive and challenging to extend to new modalities. |
| Approach: | They propose a new paradigm for multimodal large language models by reusing modality encoders and merging LLM parameters. |
| Outcome: | The proposed model retains the modal understanding capabilities of each original model. |
Graphine: A Dataset for Graph-aware Terminology Definition Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Lack of large-scale terminology definition dataset hinders definition generation . lack of precise terminology definitions poses great challenges in scientific communication . |
| Approach: | They propose a large-scale terminology definition dataset Graphine that exploits the graph structure of terminologies to generate graph-aware text generation models. |
| Outcome: | The proposed model outperforms existing models by exploiting graph structure of terminologies. |
VeriFact: Enhancing Long-Form Factuality Evaluation with Refined Fact Extraction and Reference Facts (2025.emnlp-main)
Copied to clipboard
| Challenge: | Prior work focuses on accuracy and precision, but factuality evaluation is difficult due to inter-sentence dependencies. |
| Approach: | They introduce a factuality evaluation framework to enhance fact extraction . they also introduce 'factRBench' that evaluates both precision and recall . |
| Outcome: | The proposed framework enhances fact extraction by identifying incomplete and missing facts . it also evaluates precision and recall in long-form models, whereas prior work focuses on precision. |
Revisiting De-Identification of Electronic Medical Records: Evaluation of Within- and Cross-Hospital Generalization (2023.emnlp-main)
Copied to clipboard
| Challenge: | De-identification is a natural language processing task to detect and remove the protected health information (PHI) from electronic medical records (EMRs). |
| Approach: | They propose a de-identification dataset comprising EMRs from three hospitals in China and use it to establish a benchmark for evaluating both within- and cross-hospital generalization. |
| Outcome: | The proposed model with almost perfect within-hospital performance struggles when transferred across hospitals. |
Conditional Dichotomy Quantification via Geometric Embedding (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods that rely on semantic similarity fail to capture the nuanced oppositional dynamics essential for these applications. |
| Approach: | They propose a task that formalizes the measurement of conditional dichotomy by using a dichotomian framework. |
| Outcome: | The proposed framework provides carefully constructed datasets covering debate, defeasible inference, and causal reasoning scenarios. |
SimpleOCR: Rendering Visual Questions to Teach MLLMs to Read (2026.findings-acl)
Copied to clipboard
Yibo Peng, Peng Xia, Ding Zhong, Kaide Zeng, Siwei Han, Yiyang Zhou, Jiaqi Liu, Ruiyi Zhang, Huaxiu Yao
| Challenge: | MLLMs lack visual grounding mechanism to read text embedded in images, or rely on parametric shortcuts . despite strong OCR capabilities, models suffer performance degradation of 12.7% in the VQ setting . |
| Approach: | They propose a plug-and-play training strategy that invalidates shortcuts in text prompts . they propose 'vq' setting where text queries are rendered directly onto images . |
| Outcome: | The proposed training strategy surpasses the base model by 5.4% and GRPO based on original images by 2.7% on four representative OOD benchmarks. |
M2PT: Multimodal Prompt Tuning for Zero-shot Instruction Learning (2024.emnlp-main)
Copied to clipboard
Taowen Wang, Yiyang Liu, James Liang, Junhan Zhao, Yiming Cui, Yuning Mao, Shaoliang Nie, Jiahao Liu, Fuli Feng, Zenglin Xu, Cheng Han, Lifu Huang, Qifan Wang, Dongfang Liu
| Challenge: | Multimodal Large Language Models (MLLMs) exhibit remarkable performance across a wide range of domains. |
| Approach: | They propose a multimodal prompt tuning approach for efficient instruction tuning of MLLMs. |
| Outcome: | The proposed approach shows superior performance on multimodal evaluation datasets compared to state-of-the-art methods. |
CADMate: Generating CAD Assembly Plan with Geometric Chain-of-Thought and Spatial Physical Rewards (2026.acl-long)
Copied to clipboard
| Challenge: | Computer-aided design (CAD) is crucial in prototyping complex 3D objects . designers manually define assembly sequences for individual CAD parts . |
| Approach: | They propose a framework for computer-aided design that predicts actions for CAD parts . they use a reference design image and disassembled parts to generate 6-DoF transformations . |
| Outcome: | The proposed framework outperforms existing MLLMs in the design of CAD assemblies. |
LLM-Powered Test Case Generation for Detecting Bugs in Plausible Programs (2025.acl-long)
Copied to clipboard
Kaibo Liu, Zhenpeng Chen, Yiyang Liu, Jie M. Zhang, Mark Harman, Yudong Han, Yun Ma, Yihong Dong, Ge Li, Gang Huang
| Challenge: | TrickCatcher generates test cases that pass existing tests yet contain bugs . a recent study found that tricky bugs are not detected by test suites . |
| Approach: | They propose an LLM-powered approach to generating test cases for uncovering bugs in plausible programs . they use a PUT and specification to generate program variants, an input generator and an Llm to construct test inputs . |
| Outcome: | The proposed approach achieves recall, precision, and F1 scores that are 1.80, 2.65, and 1.66 . trickCatcher generates program variants based on the program under test and its specification . |
SciCustom: A Framework for Custom Evaluation of Scientific Capabilities in Large Language Models (2026.acl-long)
Copied to clipboard
Yiyang Gu, Junwei Yang, Junyu Luo, Ye Yuan, Bin Feng, Yingce Xia, Shufang Xie, Kaili Liu, Bohan Wu, Qi Shi, Haoran Li, Beier Xiao, Zhiping Xiao, Xiao Luo, Weizhi Zhang, Philip S. Yu, Zequn Liu, Ming Zhang
| Challenge: | Existing evaluations of large language models fail to reflect fine-grained capabilities . existing benchmarks are manually curated or domain-generic, limiting scalability and alignment with real use cases. |
| Approach: | They propose a framework that allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific scientific capabilities in LLMs. |
| Outcome: | The proposed framework reveals fine-grained differences in scientific capabilities that standard benchmarks overlook . it allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific capabilities in LLMs. |
On-the-Fly VLA Adaptation via Test-Time Reinforcement Learning (2026.acl-long)
Copied to clipboard
Changyu Liu, Yiyang Liu, Taowen Wang, Qiao Zhuang, James Chenhao Liang, Wenhao Yang, Renjing Xu, Qifan Wang, Dongfang Liu, Cheng Han
| Challenge: | Existing vision-language-action models are unsuitable for simulated or physical-world deployments . current methods fail when confronted with inherent real-world dynamic variability. |
| Approach: | They propose a test-time reinforcement learning framework that enables on-the-fly policy adaptation during inference. |
| Outcome: | Empirical results show that the proposed framework improves adaptability, stability and task success in dynamic, previously unseen scenarios. |
Deep Span Representations for Named Entity Recognition (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing span-based NER systems shallowly aggregate token representations to span representations, causing performance degradations. |
| Approach: | They propose a method that shallowly aggregates token representations to span representations . DSpERT produces span representation of deep semantics with weight initialization . |
| Outcome: | The proposed model performs better than or competitive with state-of-the-art systems on six NER benchmarks. |
Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated exceptional proficiency in language-related tasks, but their deployment poses significant memory and storage requirements. |
| Approach: | They propose a method that optimizes rounding values and weight clipping within 200 steps. |
| Outcome: | The proposed method achieves exceptional results across 2 to 4 bits while maintaining low tuning costs and avoiding additional inference overhead. |
Graceful Forgetting in Generative Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained models do not provide all knowledge needed for fine-tuning tasks. |
| Approach: | They propose a framework to achieve graceful forgetting in generative language models by pre-training a model on large-scale correlating datasets. |
| Outcome: | The proposed framework improves the learning plasticity of the target task by selectively discarding irrelevant knowledge. |