Papers by Yeachan Kim
Context-based Virtual Adversarial Training for Text Classification with Noisy Labels (2022.lrec-1)
Copied to clipboard
| Challenge: | Recent studies show that deep neural networks can memorize noisy labels with limited training time. |
| Approach: | They propose a virtual adversarial training method to prevent a classifier from overfitting to noisy labels. |
| Outcome: | The proposed method performs the adversarial training in the context rather than the inputs. |
SEED: Semantic Knowledge Transfer for Language Model Adaptation to Materials Science (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods to adapt pre-trained language models to materials science rely on frequency information from limited downstream datasets. |
| Approach: | They propose a vocabulary expansion method to adapt pre-trained language models to materials science by incorporating latent materials knowledge of lightweight embeddings into PLMs. |
| Outcome: | The proposed method mitigates the limitations of existing adaptation methods and can be used in materials science. |
Towards Robust and Generalized Parameter-Efficient Fine-Tuning for Noisy Label Learning (2024.acl-long)
Copied to clipboard
| Challenge: | Parameter-efficient fine-tuning (PEFT) has enabled efficient optimization of cumbersome language models in real-world environments. |
| Approach: | They propose a routing-based PEFT approach that adaptively activates PEFT modules. |
| Outcome: | The proposed method is more sensitive to noise interference than other methods. |
KOMBO: Korean Character Representations Based on the Combination Rules of Subcharacters (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing pre-trained language models for Korean have overlooked character representation principles . a new framework for Korean PLMs uses the invention principles of Hangeul to represent letters . |
| Approach: | They propose a framework that uses Hangeul's invention principles to represent character in Korean . their method outperforms the state-of-the-art Korean PLM by 2.11% in five tasks . |
| Outcome: | The proposed method outperforms the state-of-the-art Korean PLM by 2.11% in five Korean natural language understanding tasks. |
Adaptive Compression of Word Embeddings (2020.acl-main)
Copied to clipboard
| Challenge: | Distributed representations of words have been an indispensable component for natural language processing (NLP) tasks. |
| Approach: | They propose a method that uses a code-book approach to represent words as discrete codes such as (8, 5, 2, 4). |
| Outcome: | The proposed method makes the highly compressed word embeddings without hurting the task accuracy. |
Leap-of-Thought: Accelerating Transformers via Dynamic Token Routing (2023.emnlp-main)
Copied to clipboard
| Challenge: | Inefficient transformers have been a challenge for many years, requiring computational costs that scale quadratically with the length of the input sequence. |
| Approach: | They propose a token reduction approach that dynamically routes tokens within layers to ensure that all tokens remain accessible in subsequent layers. |
| Outcome: | The proposed approach achieves up to 25x faster inference time without significant loss in accuracy. |
Representation Learning for Unseen Words by Bridging Subwords to Semantic Networks (2020.lrec-1)
Copied to clipboard
| Challenge: | Pre-trained word embeddings only include words that appeared in corpora where pre-tried embedds are learned. |
| Approach: | They propose a method to represent out-of-vocabulary words using subword information and knowledge. |
| Outcome: | The proposed method improves performance over baselines that only use subwords or knowledge to represent OOV words. |
MolTRES: Improving Chemical Language Representation Learning for Molecular Property Prediction (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for chemical representation learning often lead to overfitting and limited scalability due to early convergence. |
| Approach: | They propose a framework to train Transformers on SMILES sequences to learn from structural examples and integrate external materials embedding to enrich molecular representations. |
| Outcome: | The proposed model outperforms state-of-the-art models on molecular property prediction tasks. |
Client-Customized Adaptation for Parameter-Efficient Federated Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models have a large memory footprint and are difficult to use in federated learning (FL) |
| Approach: | They propose a hypernetwork-based FL framework that generates client-specific adapters by conditioning the client information. |
| Outcome: | The proposed framework maximizes the utility of shared model parameters while minimizing divergence caused by client heterogeneity. |
Curriculum Debiasing: Toward Robust Parameter-Efficient Fine-Tuning Against Dataset Biases (2025.acl-long)
Copied to clipboard
| Challenge: | Parameter-efficient fine-tuning (PEFT) addresses the memory footprint issue of full fine- tuning by modifying only a subset of model parameters. |
| Approach: | They propose a framework that debiases models in a biased-to-unbiased order and uses only a subset of parameters to modify model parameters. |
| Outcome: | The proposed framework accelerates convergence on unbiased examples by approximately twofold and improves ID and OOD performance by 1.2% and 8.0%, respectively. |
Bridging the Gap Between Molecule and Textual Descriptions via Substructure-aware Alignment (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing models lack the ability to learn fine-grained alignments between molecules and their descriptions. |
| Approach: | They propose a molecule–text learning framework based on substructure-aware alignments that augments original molecule-description pairs with additional alignment signals derived from molecular substructures and chemical phrases. |
| Outcome: | The proposed framework outperforms state-of-the-art models on a wide range of molecular benchmarks. |
MELT: Materials-aware Continued Pre-training for Language Model Adaptation to Materials Science (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods focused on constructing domain-specific corpus focus on a limited and scarce nature of datasets in materials science poses significant challenges for developing models that generalize well across a broad range of materials entities. |
| Approach: | They propose a method to adapt pre-trained language models for materials science by continuously pre-training them on a materials science corpus. |
| Outcome: | The proposed method is able to adapt pre-trained language models for materials science tasks. |
Moleco: Molecular Contrastive Learning with Chemical Language Models for Molecular Property Prediction (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Pre-trained chemical language models (CLMs) are highly effective in predicting molecular properties, but implicitly contain limited structural information. |
| Approach: | They propose a contrastive learning framework to enhance the understanding of molecular structures within chemical language models by measuring the similarity of fingerprint vectors among different molecules. |
| Outcome: | The proposed framework outperforms state-of-the-art models in molecular property prediction. |
Forward Knows Efficient Backward Path: Saliency-Guided Memory-Efficient Fine-tuning of Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | a number of fine-tuning approaches are available to improve performance of large language models. |
| Approach: | They propose a memory-efficient method to minimize memory associated with cached intermediate activations. |
| Outcome: | The proposed method minimizes memory associated with cached intermediate activations while preserving accuracy. |
Zero-shot Commonsense Reasoning over Machine Imagination (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent approaches to zero-shot commonsense reasoning have suffered from human reporting bias inherent in textual commonsence knowledge, leading to discrepancies in understanding between PLMs and humans. |
| Approach: | They propose a zero-shot commonsense reasoning framework that integrates machine-generated images into the reasoning process. |
| Outcome: | The proposed framework outperforms existing methods on diverse reasoning benchmarks and analysis. |
Improving Bias Mitigation through Bias Experts in Natural Language Understanding (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to mitigate the detrimental effect of bias on the network include debiasing methods that down-weight the biased examples identified by an auxiliary model, which is trained with explicit bias labels. |
| Approach: | They propose a framework that introduces binary classifiers between the auxiliary model and main model, coined bias experts, to reduce the detrimental effect of bias on the network. |
| Outcome: | The proposed approach outperforms the state-of-the-art on various datasets while achieving high performance on in-distribution data. |
SparseFlow: Accelerating Transformers by Sparsifying Information Flows (2024.acl-long)
Copied to clipboard
| Challenge: | SparseFlow is an efficient method to sparsify the dense information flows within transformers. |
| Approach: | They propose a method to sparsify the dense pathways of token representations across all transformer blocks by parameterizing them to be sparse. |
| Outcome: | The proposed method reduces computational costs by half on average without compromising task accuracy. |
Multi-pretraining for Large-scale Text Classification (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for large-scale text classification involve excessive computation and memory overheads. |
| Approach: | They propose a self-supervised and weakly supervised pretraining frameworks for large-scale text classification with multiple categories. |
| Outcome: | The proposed framework improves on the self-supervised and weakly supervised methods while being computationally efficient. |
Learning to Generate Word Representations using Subword Information (C18-1)
Copied to clipboard
| Challenge: | Existing word-based approaches to learning word representations are blind to subword information in words. |
| Approach: | They propose a character-based word representation approach to learn word representations from characters. |
| Outcome: | The proposed model outperforms baseline models that regard words as atomic units . the proposed model achieves 18.5% improvement on average in perplexity for morphologically rich languages . |