Papers with MKA
KACC: A Multi-task Benchmark for Knowledge Abstraction, Concretization and Completion (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on partial aspects of knowledge abstraction, concretization, and completion (KACC). |
| Approach: | They propose a unified knowledge graph benchmark to improve existing benchmarks . they collect new datasets that contain larger concept graphs and cross-view links . |
| Outcome: | The proposed benchmark improves existing benchmarks in terms of dataset scale, task coverage, and difficulty. |
Pruning via Merging: Compressing LLMs via Manifold Alignment Based Layer Merging (2024.emnlp-main)
Copied to clipboard
Deyuan Liu, Zhanyue Qin, Hairu Wang, Zhao Yang, Zecheng Wang, Fangying Rong, Qingbin Liu, Yanchao Hao, Bo Li, Xi Chen, Cunhang Fan, Zhao Lv, Dianhui Chu, Zhiying Tu, Dianbo Sui
| Challenge: | Existing methods for parameter pruning fail to utilize the knowledge from pruned parameters. |
| Approach: | They propose a method that uses manifold learning and the Information Bottleneck measure to merge similar layers to preserve model performance. |
| Outcome: | The proposed method outperforms pruning methods on multiple datasets and LLMs with quantization and achieves substantial compression ratios. |