Papers by Congzheng Song
Memory-Efficient Backpropagation for Fine-Tuning LLMs on Resource-Constrained Mobile Devices (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing work on memory-efficient on-device fine-tuning of large language models with backpropagation has focused on approximating gradients with zeroth-order optimization (ZO). |
| Approach: | They propose a memory-efficient implementation of backpropagation on mobile devices that allows flexible trade-offs between memory usage and compute time while converging faster. |
| Outcome: | The proposed method can fine-tune LLMs with backpropagation using less than 1GB of memory while achieving better performance than the baseline. |
Adversarial Semantic Collisions (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to generate semantic collisions for NLP tasks are vulnerable to adversarial examples. |
| Approach: | They propose gradient-based approaches for generating semantic collisions given white-box access to a model and deploy them against several NLP tasks. |
| Outcome: | The proposed approaches evade perplexity-based filtering and discuss other potential mitigations. |