Papers by SangKeun Lee
SMoP: Towards Efficient and Effective Prompt Tuning with Sparse Mixture-of-Prompts (2023.emnlp-main)
Copied to clipboard
| Challenge: | Prompt tuning has emerged as a successful parameter-efficient alternative to the full fine-tuning of language models. |
| Approach: | They propose a prompt tuning method that utilizes short soft prompts for efficient training and inference while maintaining performance gains typically induced by longer soft prompt. |
| Outcome: | The proposed method outperforms baseline methods while preserving memory usage. |
SEED: Semantic Knowledge Transfer for Language Model Adaptation to Materials Science (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods to adapt pre-trained language models to materials science rely on frequency information from limited downstream datasets. |
| Approach: | They propose a vocabulary expansion method to adapt pre-trained language models to materials science by incorporating latent materials knowledge of lightweight embeddings into PLMs. |
| Outcome: | The proposed method mitigates the limitations of existing adaptation methods and can be used in materials science. |
Towards Robust and Generalized Parameter-Efficient Fine-Tuning for Noisy Label Learning (2024.acl-long)
Copied to clipboard
| Challenge: | Parameter-efficient fine-tuning (PEFT) has enabled efficient optimization of cumbersome language models in real-world environments. |
| Approach: | They propose a routing-based PEFT approach that adaptively activates PEFT modules. |
| Outcome: | The proposed method is more sensitive to noise interference than other methods. |
“Why do I feel offended?” - Korean Dataset for Offensive Language Identification (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for detecting offensive content rely on labeled datasets, but few consider low-resource languages with relatively less data available for training. |
| Approach: | They propose to use Korean as a dataset for offensive language identification . they propose to perform abusive language detection and sentiment analysis to help identify offensive languages. |
| Outcome: | The proposed datasets improve the performance of offensive language identification in Korean, while the existing methods are limited. |
DIVE: Towards Descriptive and Diverse Visual Commonsense Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Towards human-level visual understanding, visual commonsense generation has been introduced . but current research on visual commonense generation ignores an important human cognitive ability . |
| Approach: | They propose a visual commonsense generation framework to improve inferences by visual common sense generation. |
| Outcome: | The proposed framework outperforms state-of-the-art models in descriptiveness and diversity . human evaluations confirm that the framework aligns closely with human judgments on descriptiveness . |
Mentor-KD: Making Small Language Models Better Multi-step Reasoners (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown impressive emergent capabilities by leveraging Chain-of-Thought (CoT) prompting. |
| Approach: | They propose a Knowledge Distillation approach which transfers multi-step reasoning ability of Large Language Models (LLMs) to smaller LMs by fine-tuning language models of multi- step rationales generated by LLM teachers. |
| Outcome: | The proposed method is able to transfer multi-step reasoning ability of LLMs to smaller LMs while addressing data quality and soft label provision. |
Adaptive Convolution for Text Classification (N19-1)
Copied to clipboard
| Challenge: | Existing convolutional neural networks (CNNs) use sparse representations of text, such as bag-of-words. |
| Approach: | They propose an adaptive convolution for text classification to give flexibility to convolutional neural networks (CNNs) they attach filter-generating networks to convevolution blocks in existing CNNs . |
| Outcome: | The proposed convolution improves performance in seven benchmark datasets by 2.6 percentage points . the proposed conversions can be likened to players of the twenty questions . |
Break it Down into BTS: Basic, Tiniest Subword Units for Korean (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing word embeddings for Korean use the internal structure of words with subword information to improve the quality of word representations. |
| Approach: | They introduce Basic, Tiniest Subword (BTS) units for Korean language that are inspired by Hangeul, the Korean writing system. |
| Outcome: | The proposed framework outperforms the state-of-the-art Korean word embedding by 11.8% on all intrinsic and extrinsic tasks. |
PICTURE: Enhancing Theory-of-Mind in Large Language Models by Revealing, Not Hiding, Characters’ Lack of Knowledge (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to simulating Theory of Mind (ToM) using LLMs have been a longstanding problem in natural language processing (NLP). |
| Approach: | They propose a new method that enables LLMs to generate a character’s lack of knowledge within free-form Chain-of-Thought (CoT) based on this method, they propose to generate perspective-taking outputs as free- form explanations without event hiding. |
| Outcome: | The proposed method outperforms existing prompting methods by an average of 7.3% on false-belief tasks. |
KOMBO: Korean Character Representations Based on the Combination Rules of Subcharacters (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing pre-trained language models for Korean have overlooked character representation principles . a new framework for Korean PLMs uses the invention principles of Hangeul to represent letters . |
| Approach: | They propose a framework that uses Hangeul's invention principles to represent character in Korean . their method outperforms the state-of-the-art Korean PLM by 2.11% in five tasks . |
| Outcome: | The proposed method outperforms the state-of-the-art Korean PLM by 2.11% in five Korean natural language understanding tasks. |
Adaptive Compression of Word Embeddings (2020.acl-main)
Copied to clipboard
| Challenge: | Distributed representations of words have been an indispensable component for natural language processing (NLP) tasks. |
| Approach: | They propose a method that uses a code-book approach to represent words as discrete codes such as (8, 5, 2, 4). |
| Outcome: | The proposed method makes the highly compressed word embeddings without hurting the task accuracy. |
Coconut: Contextualized Commonsense Unified Transformers for Graph-Based Commonsense Augmentation of Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that pre-trained language models lack commonsense knowledge . |
| Approach: | They propose a contextualized knowledge prompting scheme to guide the contextualization of structured commonsense knowledge based on large language models. |
| Outcome: | The proposed approach outperforms the state-of-the-art technique by an average of 5.8%. |
Leap-of-Thought: Accelerating Transformers via Dynamic Token Routing (2023.emnlp-main)
Copied to clipboard
| Challenge: | Inefficient transformers have been a challenge for many years, requiring computational costs that scale quadratically with the length of the input sequence. |
| Approach: | They propose a token reduction approach that dynamically routes tokens within layers to ensure that all tokens remain accessible in subsequent layers. |
| Outcome: | The proposed approach achieves up to 25x faster inference time without significant loss in accuracy. |
Incorporating Domain Knowledge into Materials Tokenization (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in language models have expanded their applications in materials science, but they often produce excessive fragmentation and semantic loss. |
| Approach: | They propose a frequency-centric tokenization approach that integrates material knowledge into tokenization. |
| Outcome: | The proposed tokenization approach outperforms existing tokenization methods and achieves an average performance gain of 4% and 2% in the generation and classification tasks. |
Representation Learning for Unseen Words by Bridging Subwords to Semantic Networks (2020.lrec-1)
Copied to clipboard
| Challenge: | Pre-trained word embeddings only include words that appeared in corpora where pre-tried embedds are learned. |
| Approach: | They propose a method to represent out-of-vocabulary words using subword information and knowledge. |
| Outcome: | The proposed method improves performance over baselines that only use subwords or knowledge to represent OOV words. |
Tutoring Helps Students Learn Better: Improving Knowledge Distillation for BERT with Tutor Network (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge distillation approaches for language models have overlooked the difficulty of training examples. |
| Approach: | They propose a framework that controls difficulty of training examples during pre-training by a tutor network. |
| Outcome: | The proposed framework outperforms state-of-the-art KD methods with student models on the GLUE benchmark. |
MolTRES: Improving Chemical Language Representation Learning for Molecular Property Prediction (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for chemical representation learning often lead to overfitting and limited scalability due to early convergence. |
| Approach: | They propose a framework to train Transformers on SMILES sequences to learn from structural examples and integrate external materials embedding to enrich molecular representations. |
| Outcome: | The proposed model outperforms state-of-the-art models on molecular property prediction tasks. |
Client-Customized Adaptation for Parameter-Efficient Federated Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models have a large memory footprint and are difficult to use in federated learning (FL) |
| Approach: | They propose a hypernetwork-based FL framework that generates client-specific adapters by conditioning the client information. |
| Outcome: | The proposed framework maximizes the utility of shared model parameters while minimizing divergence caused by client heterogeneity. |
Curriculum Debiasing: Toward Robust Parameter-Efficient Fine-Tuning Against Dataset Biases (2025.acl-long)
Copied to clipboard
| Challenge: | Parameter-efficient fine-tuning (PEFT) addresses the memory footprint issue of full fine- tuning by modifying only a subset of model parameters. |
| Approach: | They propose a framework that debiases models in a biased-to-unbiased order and uses only a subset of parameters to modify model parameters. |
| Outcome: | The proposed framework accelerates convergence on unbiased examples by approximately twofold and improves ID and OOD performance by 1.2% and 8.0%, respectively. |
Bridging the Gap Between Molecule and Textual Descriptions via Substructure-aware Alignment (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing models lack the ability to learn fine-grained alignments between molecules and their descriptions. |
| Approach: | They propose a molecule–text learning framework based on substructure-aware alignments that augments original molecule-description pairs with additional alignment signals derived from molecular substructures and chemical phrases. |
| Outcome: | The proposed framework outperforms state-of-the-art models on a wide range of molecular benchmarks. |
Learning from Missing Relations: Contrastive Learning with Commonsense Knowledge Graphs for Commonsense Inference (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to commonsense inference lack coverage and expressive diversity of commonsensense knowledge graphs. |
| Approach: | They propose a framework that contrasts sets of semantically similar and dissimilar events . they propose 'solar' framework that can be used to learn commonsense inference . |
| Outcome: | The proposed framework outperforms the state-of-the-art commonsense transformer on commonsensense inference by 1.84% on average among 8 metrics. |
MELT: Materials-aware Continued Pre-training for Language Model Adaptation to Materials Science (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods focused on constructing domain-specific corpus focus on a limited and scarce nature of datasets in materials science poses significant challenges for developing models that generalize well across a broad range of materials entities. |
| Approach: | They propose a method to adapt pre-trained language models for materials science by continuously pre-training them on a materials science corpus. |
| Outcome: | The proposed method is able to adapt pre-trained language models for materials science tasks. |
Moleco: Molecular Contrastive Learning with Chemical Language Models for Molecular Property Prediction (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Pre-trained chemical language models (CLMs) are highly effective in predicting molecular properties, but implicitly contain limited structural information. |
| Approach: | They propose a contrastive learning framework to enhance the understanding of molecular structures within chemical language models by measuring the similarity of fingerprint vectors among different molecules. |
| Outcome: | The proposed framework outperforms state-of-the-art models in molecular property prediction. |
Forward Knows Efficient Backward Path: Saliency-Guided Memory-Efficient Fine-tuning of Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | a number of fine-tuning approaches are available to improve performance of large language models. |
| Approach: | They propose a memory-efficient method to minimize memory associated with cached intermediate activations. |
| Outcome: | The proposed method minimizes memory associated with cached intermediate activations while preserving accuracy. |
KOAS: Korean Text Offensiveness Analysis System (2021.emnlp-demo)
Copied to clipboard
San-Hee Park, Kang-Min Kim, Seonhee Cho, Jun-Hyung Park, Hyuntae Park, Hyuna Kim, Seongwon Chung, SangKeun Lee
| Challenge: | morphological richness and complex syntax of Korean cause difficulties in neural model training. |
| Approach: | They propose a system that exploits contextual and linguistic features and estimates an offensiveness score for a Korean text. |
| Outcome: | The proposed system exploits both contextual and linguistic features and estimates an offensiveness score for a Korean text. |
Zero-shot Commonsense Reasoning over Machine Imagination (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent approaches to zero-shot commonsense reasoning have suffered from human reporting bias inherent in textual commonsence knowledge, leading to discrepancies in understanding between PLMs and humans. |
| Approach: | They propose a zero-shot commonsense reasoning framework that integrates machine-generated images into the reasoning process. |
| Outcome: | The proposed framework outperforms existing methods on diverse reasoning benchmarks and analysis. |
Polishing Every Facet of the GEM: Testing Linguistic Competence of LLMs and Humans in Korean (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies have focused on linguistic competence of language models with grammatical knowledge. |
| Approach: | They propose to use grammar as a measurable proxy to assess linguistic competence of large language models (LLMs) . |
| Outcome: | The proposed model aims to assess the linguistic competence of large language models (LLMs) and humans in Korean. |
“Going to a trap house” conveys more fear than “Going to a mall”: Benchmarking Emotion Context Sensitivity for LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new benchmark evaluates whether large language models can understand emotion context sensitivity of humans. |
| Approach: | a new benchmark evaluates whether large language models can understand emotion context sensitivity of humans. |
| Outcome: | a new benchmark evaluates whether large language models can understand emotion context sensitivity of humans. |
Improving Bias Mitigation through Bias Experts in Natural Language Understanding (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to mitigate the detrimental effect of bias on the network include debiasing methods that down-weight the biased examples identified by an auxiliary model, which is trained with explicit bias labels. |
| Approach: | They propose a framework that introduces binary classifiers between the auxiliary model and main model, coined bias experts, to reduce the detrimental effect of bias on the network. |
| Outcome: | The proposed approach outperforms the state-of-the-art on various datasets while achieving high performance on in-distribution data. |
SCRIPT: A Subcharacter Compositional Representation Injection Module for Korean Pre-Trained Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Korean uses a featural writing system in which each character is composed of subcharacter units known as Jamo. |
| Approach: | They propose a model-agnostic module that injects subcharacter compositional knowledge into Korean language models. |
| Outcome: | a new module improves embeddings of Korean subwords with structural granularity . the module improve grammatical regularities and semantic cohesive variations . |
Efficient Pre-training of Masked Language Model via Concept-based Curriculum Masking (2022.emnlp-main)
Copied to clipboard
| Challenge: | Masked language modeling (MLM) has been widely used for pre-training effective bidirectional representations but comes at a substantial training cost. |
| Approach: | They propose a concept-based curriculum masking method that evaluates the MLM difficulty of each token based on a carefully-designed linguistic difficulty criterion. |
| Outcome: | The proposed method significantly improves pre-training efficiency with the original BERT model at half the training cost. |
SparseFlow: Accelerating Transformers by Sparsifying Information Flows (2024.acl-long)
Copied to clipboard
| Challenge: | SparseFlow is an efficient method to sparsify the dense information flows within transformers. |
| Approach: | They propose a method to sparsify the dense pathways of token representations across all transformer blocks by parameterizing them to be sparse. |
| Outcome: | The proposed method reduces computational costs by half on average without compromising task accuracy. |
Multi-pretraining for Large-scale Text Classification (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for large-scale text classification involve excessive computation and memory overheads. |
| Approach: | They propose a self-supervised and weakly supervised pretraining frameworks for large-scale text classification with multiple categories. |
| Outcome: | The proposed framework improves on the self-supervised and weakly supervised methods while being computationally efficient. |
Connecting the Knowledge Dots: Retrieval-augmented Knowledge Connection for Commonsense Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show that large language models exhibit a limited understanding of commonsense reasoning due to the necessity of implicit knowledge that is rarely expressed in text. |
| Approach: | They propose a retrieval-augmented knowledge connection framework that transforms indirectly relevant documents into a direct explanation to answer a given question. |
| Outcome: | The proposed framework outperforms state-of-the-art (SOTA) benchmarks and achieves +2.0% and +4.6% average accuracy on in-domain (ID) and out-of domain (OOD) benchmark. |
Learning to Generate Word Representations using Subword Information (C18-1)
Copied to clipboard
| Challenge: | Existing word-based approaches to learning word representations are blind to subword information in words. |
| Approach: | They propose a character-based word representation approach to learn word representations from characters. |
| Outcome: | The proposed model outperforms baseline models that regard words as atomic units . the proposed model achieves 18.5% improvement on average in perplexity for morphologically rich languages . |
Handling Out-Of-Vocabulary Problem in Hangeul Word Embeddings (2021.eacl-main)
Copied to clipboard
| Challenge: | Word embedding is considered an essential factor in improving the performance of various Natural Language Processing (NLP) models. |
| Approach: | They propose a Hangeul word embedding model that infers original word embeds from typos while maintaining high performance. |
| Outcome: | The proposed model performs well against typos while maintaining high performance. |