Papers by Zili Zhou
Multi-matrix Factorization Attention (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing variants for Multi-Head Attention (MHA) fail to maintain strong performance under stringent Key-Value cache (KV cache) constraints. |
| Approach: | They propose to use multi-matrix factorization attention and MFA-Key-reuse attention architectures to increase model capacity under tight KV cache constraints. |
| Outcome: | The proposed architecture outperforms existing methods while reducing KV cache usage by 56% and 93.7% in large-scale experiments. |
PhysNLU: A Language Resource for Evaluating Natural Language Understanding and Explanation Coherence in Physics (2022.lrec-1)
Copied to clipboard
| Challenge: | physicists use mathematics to reason and explain, separates their field from other disciplines, including mathematics. |
| Approach: | They present a dataset to evaluate the performance of language models in physics . they find that language models are challenged by coherence related tasks in physicists . |
| Outcome: | The proposed models are able to perform well on coherence-related tasks even when trained on natural language objectives. |
SSEGCN: Syntactic and Semantic Enhanced Graph Convolutional Network for Aspect-based Sentiment Analysis (2022.naacl-main)
Copied to clipboard
| Challenge: | Aspect-based Sentiment Analysis (ABSA) aims to predict sentiment polarity towards aspects in sentences . a novel model for ABSA is proposed, but how to harness it is still a challenge . |
| Approach: | They propose a syntactic and semantic enhanced Graph Convolutional Network (SSEGCN) model for ABSA task using aspect-aware attention mechanism and self-attention. |
| Outcome: | The proposed model outperforms state-of-the-art methods on benchmark datasets. |
ChatMusician: Understanding and Generating Music Intrinsically with LLM (2024.findings-acl)
Copied to clipboard
Ruibin Yuan, Hanfeng Lin, Yi Wang, Zeyue Tian, Shangda Wu, Tianhao Shen, Ge Zhang, Yuhang Wu, Cong Liu, Ziya Zhou, Liumeng Xue, Ziyang Ma, Qin Liu, Tianyu Zheng, Yizhi Li, Yinghao Ma, Yiming Liang, Xiaowei Chi, Ruibo Liu, Zili Wang, Chenghua Lin, Qifeng Liu, Tao Jiang, Wenhao Huang, Wenhu Chen, Jie Fu, Emmanouil Benetos, Gus Xia, Roger Dannenberg, Wei Xue, Shiyin Kang, Yike Guo
| Challenge: | Despite LLMs' impressive capabilities in musical knowledge, music reasoning remains an unsolved task. |
| Approach: | They propose an open-source large language model (LLM) that integrates intrinsic musical abilities into LLaMA2 and GPT-3.5. |
| Outcome: | The proposed model can understand and generate music with a pure text tokenizer without external multi-modal neural structures or tokenizers. |
COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for Alignment with Human Values (2026.findings-eacl)
Copied to clipboard
Siwei Wu, JinCheng Ren, Xeron Du, Shuyue Guo, Xingwei Qu, Yiming Liang, Jie Liu, Yunwen Li, Tyler Loakman, Tianyu Zheng, Boyu Feng, Huaqing Yuan, Zili Wang, Jiaheng Liu, Wenhao Huang, Chenglin Cai, Haoran Que, Jian Yang, Yuelin Bai, Zekun Moore Wang, Zhouliang Yu, Qunshu Lin, Ding Pan, Yuchen Eleanor Jiang, Tiannan Wang, Wangchunshu Zhou, Shenzhi Wang, Xingyuan Bu, Minghao Liu, Guoyin Wang, Ge Zhang, Chenghua Lin
| Challenge: | Existing Chinese preference datasets suffer from limited scale, restricted domain coverage, and insufficiently rigorous data validation. |
| Approach: | They propose an LLM-based data annotation pipeline with no human intervention to annotate Chinese preference datasets. |
| Outcome: | The proposed pipeline outperforms existing Chinese preference datasets on AlignBench and Chinese Reward Benchmark. |