Papers by Wooin Lee
SAGE: Sign-Adaptive Gradient for Memory-Efficient LLM Optimization (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to train LLMs consume memory equivalent to twice the model size, resulting in a hybrid design that reverts to AdamW and negates the memory gains. |
| Approach: | They propose a new, memory-efficient O(d) adaptive scale that replaces AdamW in a hybrid structure that combines a Lion-style update direction with a memory-saving adaptive scale. |
| Outcome: | The proposed model outperforms existing methods on LLMs up to 1.3B parameters while significantly reducing optimizer state memory. |
CoMPM: Context Modeling with Speaker’s Pre-trained Memory Tracking for Emotion Recognition in Conversation (2022.naacl-main)
Copied to clipboard
| Challenge: | Emotion recognition in conversation is inaccurate if the previous utterances are not taken into account, so many studies reflect the dialogue context to improve the performance. |
| Approach: | They propose a method that combines pre-trained memory with the context model to improve the performance of the context models. |
| Outcome: | The proposed method achieves the first or second performance on all data and is state-of-the-art among systems that do not leverage structured data. |