Papers by Gongfan Fang
CoT-Valve: Length-Compressible Chain-of-Thought Tuning (2025.acl-long)
Copied to clipboard
| Challenge: | Wei et al., 2022) have developed a powerful method for enhancing the reasoning capabilities of large language models. |
| Approach: | They propose to use a tuning and inference strategy to control the length of reasoning chains by a parameter space direction to control their length. |
| Outcome: | The proposed method reduces reasoning chains on GSM8K from 741 to 225 tokens with a minor performance drop (95.07% to 94.92%) and on AIME from 6827 to 4629 tokens, with only one additional incorrect answer. |
Adversarial Self-Supervised Data-Free Distillation for Text Classification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge distillation algorithms rely on the accessibility of the training dataset, which may be unavailable due to privacy issues. |
| Approach: | They propose a data-free distillation method for a pre-trained transformer-based model that uses plug & play Embedding Guessing to craft pseudo embeddings from the teacher's hidden knowledge. |
| Outcome: | The proposed method is the first data-free distillation framework designed for NLP tasks. |