Papers by Hanyu Hu
FISTAPruner: Layer-wise Post-training Pruning for Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing pruning methods require inefficient retraining for billion-scale LLMs or rely on heuristicically designed metrics to determine pruning masks, leading to performance degradation. |
| Approach: | They propose a convex optimization model that induces sparsity in large language models by leveraging FISTA. |
| Outcome: | The proposed method can remove 50% of model parameters while retaining 98.6% and 95.6% of the zero-shot performance. |
Controlled Text Generation for Large Language Model with Dynamic Attribute Graphs (2024.findings-acl)
Copied to clipboard
| Challenge: | Controlled Text Generation (CTG) aims to produce texts that exhibit specific desired attributes. |
| Approach: | They propose a pluggable CTG framework for Large Language Models to control text . they use attribute scorers to evaluate attributes of sentences and construct dynamic attribute graphs . |
| Outcome: | The proposed framework achieves a peak improvement of 19.29% over baseline methods in two tasks. |
Knowledgeable Parameter Efficient Tuning Network for Commonsense Question Answering (2023.acl-long)
Copied to clipboard
| Challenge: | Existing commonsense question answering models incur prohibitive computation costs and poor interpretability . |
| Approach: | They propose a parameter efficient tuning network to pair PLMs with external knowledge for commonsense question answering. |
| Outcome: | The proposed adapter integrates entity- and query-related knowledge at a small cost. |