Papers by Jaeseong Lee
TAGQuant: Token-Aware Clustering for Group-Wise Quantization (2026.eacl-industry)
Copied to clipboard
| Challenge: | Existing work clusters channels using token dimension, which is suboptimal for grouping . a common challenge in LLM quantization is supporting "group-wise" quantization . |
| Approach: | They propose a method to group channels with similar activation distributions using tokens . they propose shuffle operation that reduces relative GSM8K error by 86% . |
| Outcome: | The proposed method reduces GSM8K error by 86% in both INT4 and MXFP4 formats compared to baselines . |
Automatic Mathematic In-Context Example Generation for LLM Using Multi-Modal Consistency (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for in-context learning require annotated datasets, resulting in higher computational costs and lower quality examples. |
| Approach: | They propose a framework that automatically generates high-quality in-context examples to enhance LLMs’ mathematical reasoning. |
| Outcome: | Evaluated on four math problem datasets, the proposed framework outperforms baseline methods with LLM accuracy ranging from 87.0% to 99.3%. |
ScriptMix: Mixing Scripts for Low-resource Language Parsing (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing work has considered transliteration and vocabulary augmentation, but the consideration of combining the two has been lacking. |
| Approach: | They propose a multilingual pretrained language model that combines two strengths and overcomes the hurdle of combining them. |
| Outcome: | The proposed model improves POS accuracy by 14% and improves DEP LAS score by 5.6%. |
Breaking ReLU Barrier: Generalized MoEfication for Dense Pretrained Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to convert pretrained dense models to MoEs are limited to ReLU-based models with natural sparsity. |
| Approach: | They propose a G-MoEfication approach for arbitrary dense models where activation sparsity assumptions no longer hold. |
| Outcome: | The proposed method reduces the inference cost associated with dense models by sparsely activating experts. |
COMMIT: Code-Mixing English-Centric Large Language Model for Multilingual Instruction Tuning (2024.findings-naacl)
Copied to clipboard
| Challenge: | COMMIT improves the exact match score of low-resource language QA by 32x. |
| Approach: | They propose to specialize instruction tuning to deviate from English-centric instruction tuning . they propose to perform cross-lingual alignment to overcome data imbalance . |
| Outcome: | The proposed method improves the exact match score of low-resource language QA by 32x. |
Interventional Speech Noise Injection for ASR Generalizable Spoken Language Understanding (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to increase the robustness of pre-trained language models (PLMs) against unseen ASR systems produce noisy inputs for SLU models, which can significantly degrade their performance. |
| Approach: | They propose to introduce ASR-plausible noises into pre-trained language models by cutting off the non-causal effect of noises. |
| Outcome: | The proposed method improves the robustness and generalizability of SLU models against unseen ASR systems by cutting off the non-causal effect of noises. |
Multilingual Lottery Tickets to Pretrain Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Multilingual pretrained language models (mPLMs) have become the de-facto standard for multilingual tasks. |
| Approach: | They propose to search for a per-language subnetwork with comparable performance to the full model by scaling the model to reduce interference and then redistributing parameters to keep the parameters reduced. |
| Outcome: | The proposed model reduces the inference cost of models for each language while keeping the capacity per language more or less the same. |
PROM: Pivoted and Regulated Optimization for Multilingual Instruction Learning (2025.naacl-short)
Copied to clipboard
| Challenge: | Existing solutions to large language models (LLMs) are English-centric, hindering their application to 6500+ existing languages. |
| Approach: | They propose to append English tuning data with its translated pair to solve this problem . they identify English as an internal pivot language and propose to regulate between them . |
| Outcome: | The proposed model is able to generalize on multiple benchmarks across different languages. |
ContrastiveMix: Overcoming Code-Mixing Dilemma in Cross-Lingual Transfer for Information Retrieval (2024.naacl-short)
Copied to clipboard
| Challenge: | Multilingual pretrained language models have been widely adopted in cross-lingual transfer . however, training mPLMs on code-mixed data is counterproductive . |
| Approach: | They propose a method that disentangles contrastive loss between conflicting objectives . they propose mPLMs that train on code-mixed data for information retrieval . |
| Outcome: | The proposed model outperforms mDPR in cross-lingual transfer tasks . it is effective in sentence-level and token-level zero-shot tasks despite lack of target language data . |
STUN: Structured-Then-Unstructured Pruning for Scalable MoE Pruning (2025.acl-long)
Copied to clipboard
| Challenge: | Mixture-of-experts (MoEs) have been adopted for reducing inference costs by sparsely activating experts in large language models (LLMs). |
| Approach: | They propose a structured-then-unstructured approach outperforming both of structured and unstructured pruning for MoEs. |
| Outcome: | The proposed approach outperforms both of structured and unstructured pruning, especially for MoEs with hundreds of experts. |
Smarter, Not Harder: Training-Free Adaptive Computation for Transformers (2025.findings-acl)
Copied to clipboard
| Challenge: | Adaptive Computation in Transformers (ACT) aims to reduce computation for simpler inferences while enhancing performance by allocating more computation to complex inference steps. |
| Approach: | They propose a method that perturbs network weights rather than input embeddings to improve performance. |
| Outcome: | The proposed method outperforms beam search and hesitation-based methods but suffers from inefficiency and instability due to its reliance on randomness. |
FAD-X: Fusing Adapters for Cross-lingual Transfer to Low-Resource Languages (2022.aacl-short)
Copied to clipboard
| Challenge: | Adapter-based tuning is a technique that selectively updates language-specific parameters to adapt to a new language, rather than fine-tuning all shared weights. |
| Approach: | They propose to add light-weight adapters to multilingual pretrained language models (mPLMs) and add language-specific parameters to adapt to a new language. |
| Outcome: | The proposed adapter can enhance cross-lingual transfer from pretrained adapters for well-known named entity recognition and classification benchmarks. |
PLEX: Adaptive Parameter-Efficient Fine-Tuning for Code LLMs using Lottery-Tickets (2025.naacl-industry)
Copied to clipboard
| Challenge: | PLEX is a lottery-ticket based parameter-efficient fine-tuning method that adapts large language models to well-supported and underrepresented programming languages (PLs) in pretraining. |
| Approach: | They propose a lottery-ticket based parameter-efficient fine-tuning method that adapts large language models to well-supported and underrepresented programming languages (PLs) |
| Outcome: | The proposed method achieves state-of-the-art performance among PEFT methods while maintaining competitive results with reduced computational overhead. |