Papers by Jaeho Lee
Prefixing Attention Sinks can Mitigate Activation Outliers for Large Language Model Quantization (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in activation quantization methods cause outliers in tokens, causing extra overhead and speedup . a method to quantize per-tensor activation is currently challenging due to the outlier activation outlier. |
| Approach: | They propose a method to find a set of key-value cache which mitigates outliers in subsequent tokens when inserted as a prefix. |
| Outcome: | The proposed method surpasses the established baseline of per-tensor activation quantization and can be seamlessly integrated with the recent activation quantitative method. |
Riemannian Optimization for LoRA on the Stiefel Manifold (2025.findings-emnlp)
Copied to clipboard
| Challenge: | powerful, large language models (LLMs) present significant fine-tuning challenges due to their size. |
| Approach: | They propose to optimize LoRA’s B matrix by imposing explicit orthogonality constraints that achieve near-perfect orthogonal and full effective rank. |
| Outcome: | The proposed method outperforms AdamW and LoRA in terms of parameter efficiency and representational capacity. |
Lost in the Prompt Order: Revealing the Limitations of Causal Attention in Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models exhibit surprising sensitivity to structure of the prompt, but mechanisms underlying this sensitivity remain poorly understood. |
| Approach: | They conduct an in-depth investigation on placing context before the questions and options in MCQA prompts. |
| Outcome: | The proposed model outperforms the reverse order (QOC) by over 14%p over a wide range of models and datasets. |
Rethinking Pruning Large Language Models: Benefits and Pitfalls of Reconstruction Error Minimization (2024.emnlp-main)
Copied to clipboard
| Challenge: | minimizing reconstruction error is not always ideal and can overfit calibration data. |
| Approach: | They propose a method to prune large language models by divide and conquer . they propose minimizing reconstruction error by more than 90% by using calibration data . |
| Outcome: | The proposed pruning approach generates high reconstruction errors . the proposed technique reduces reconstruction error by more than 90% . |
SCANNER: Knowledge-Enhanced Approach for Robust Multi-modal Named Entity Recognition of Unseen Entities (2024.naacl-long)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a task to identify textual spans that correspond to named entities in the given text. |
| Approach: | They propose a model that can generalize to entities unseen during training and handle noisy annotations. |
| Outcome: | The proposed model outperforms existing methods on both MNER and GMNER benchmarks and is robust and accurate. |
RRADistill: Distilling LLMs’ Passage Ranking Ability for Long-Tail Queries Document Re-Ranking on a Search Engine (2024.emnlp-industry)
Copied to clipboard
Nayoung Choi, Youngjune Lee, Gyu-Hwung Cho, Haeyu Jeong, Jungmin Kong, Saehun Kim, Keunchan Park, Sarah Cho, Inchang Jeong, Gyohee Nam, Sunghoon Han, Wonil Yang, Jaeho Choi
| Challenge: | Large Language Models excel at understanding the semantic relationships between queries and documents, even with lengthy and complex long-tail queries. |
| Approach: | They propose an efficient label generation pipeline and novel sLLM training methods for both encoder and decoder models. |
| Outcome: | The proposed method improves re-ranking for long-tail queries on a Korean-based search platform. |
Decoding with Limited Teacher Supervision Requires Understanding When to Trust the Teacher (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated their tremendous capability to generate human-like text sentences that convey rich knowledge in various problem domains. |
| Approach: | They propose an algorithm to aggregate small-scale LLM and LLM predictions on initial tokens so that the generated tokens can more accurately condition the subsequent token generation by small-level LLM only. |
| Outcome: | The proposed method improves on the limited supervision scenario on a wide range of models and datasets. |
Imagine to Hear: Auditory Knowledge Generation can be an Effective Assistant for Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to augment language models with audio databases lack the ability to understand auditory signals like humans. |
| Approach: | They propose a method that augments language models with audio databases to generate auditory knowledge . their framework detects multiple audio-related textual spans from the given prompt . |
| Outcome: | The proposed approach achieves state-of-the-art performance on AuditoryBench without external databases. |
Speculative End-Turn Detector for Efficient Speech Chatbot Assistant (2026.acl-long)
Copied to clipboard
| Challenge: | Spoken dialogue systems with large language models struggle with end-turn detection . this limitation often leads to premature or delayed responses, disrupting the flow of spoken conversations. |
| Approach: | They propose a dataset for end-turn detection that uses a lightweight GRU-based model and a high-performance Wav2vec-based system to make a more challenging classification of distinguishing turn ends from mere pauses. |
| Outcome: | The proposed framework significantly improves real-time ETD accuracy while keeping computations low. |
Overlapping Context with Variable-Length Stride Increases Diversity when Training Large Language Model for Code (2025.acl-industry)
Copied to clipboard
Geonmo Gu, Jaeho Kwak, Haksoo Moon, Hyun Seung Shim, Yu Jin Kim, Byoungjip Kim, Moontae Lee, Hyejeong Jeon
| Challenge: | Large language models for code (LLMs) are gaining more and more attention due to their wide applicability. |
| Approach: | They propose a method which extracts overlapping contexts from training data using variable-length stride. |
| Outcome: | The proposed method outperforms the conventional approach of controlling the number of epochs in terms of the pass@k rate. |
Towards Federated Low-Rank Adaptation of Language Models with Rank Heterogeneity (2025.naacl-short)
Copied to clipboard
| Challenge: | Low-rank adaptation (LoRA) is an efficient alternative to full-weight adaptation in federated fine-tuning of language models, significantly reducing computational costs. |
| Approach: | They propose a low-rank adaptation method that freezes original weights and trains only the update parametrized as a product of two low-ranked matrices. |
| Outcome: | The proposed method accelerates convergence and enhances the global model’s predictive performance. |
Memory-Efficient Structured Backpropagation for On-Device LLM Fine-Tuning (2026.acl-industry)
Copied to clipboard
| Challenge: | Existing approaches for fine-tuning large language models require a trade-off between exact gradients with high memory and low memory with noisy estimates (MeZO). |
| Approach: | They propose a method which derivates gradients from LoRA's low-rank structure and manually deriving backward passes to exploit the low-level structure. |
| Outcome: | The proposed method reduces peak memory from 361MB to 136MB for Qwen2.5-0.5B, enabling fine-tuning scenarios previously infeasible on memory-constrained devices. |