Papers by Junho Park
SMoP: Towards Efficient and Effective Prompt Tuning with Sparse Mixture-of-Prompts (2023.emnlp-main)
Copied to clipboard
| Challenge: | Prompt tuning has emerged as a successful parameter-efficient alternative to the full fine-tuning of language models. |
| Approach: | They propose a prompt tuning method that utilizes short soft prompts for efficient training and inference while maintaining performance gains typically induced by longer soft prompt. |
| Outcome: | The proposed method outperforms baseline methods while preserving memory usage. |
PRISP: Privacy-Safe Few-Shot Personalization via Lightweight Adaptation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for large language model personalization are limited by data-rich settings and privacy risks. |
| Approach: | They propose a lightweight and privacy-safe personalization framework tailored to constraints in large language models. |
| Outcome: | Experiments on a few-shot variant of the LaMP benchmark show that PRISP achieves strong overall performance compared to prior approaches. |
Coconut: Contextualized Commonsense Unified Transformers for Graph-Based Commonsense Augmentation of Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that pre-trained language models lack commonsense knowledge . |
| Approach: | They propose a contextualized knowledge prompting scheme to guide the contextualization of structured commonsense knowledge based on large language models. |
| Outcome: | The proposed approach outperforms the state-of-the-art technique by an average of 5.8%. |
Leap-of-Thought: Accelerating Transformers via Dynamic Token Routing (2023.emnlp-main)
Copied to clipboard
| Challenge: | Inefficient transformers have been a challenge for many years, requiring computational costs that scale quadratically with the length of the input sequence. |
| Approach: | They propose a token reduction approach that dynamically routes tokens within layers to ensure that all tokens remain accessible in subsequent layers. |
| Outcome: | The proposed approach achieves up to 25x faster inference time without significant loss in accuracy. |
Incorporating Domain Knowledge into Materials Tokenization (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in language models have expanded their applications in materials science, but they often produce excessive fragmentation and semantic loss. |
| Approach: | They propose a frequency-centric tokenization approach that integrates material knowledge into tokenization. |
| Outcome: | The proposed tokenization approach outperforms existing tokenization methods and achieves an average performance gain of 4% and 2% in the generation and classification tasks. |
Tutoring Helps Students Learn Better: Improving Knowledge Distillation for BERT with Tutor Network (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge distillation approaches for language models have overlooked the difficulty of training examples. |
| Approach: | They propose a framework that controls difficulty of training examples during pre-training by a tutor network. |
| Outcome: | The proposed framework outperforms state-of-the-art KD methods with student models on the GLUE benchmark. |
Client-Customized Adaptation for Parameter-Efficient Federated Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models have a large memory footprint and are difficult to use in federated learning (FL) |
| Approach: | They propose a hypernetwork-based FL framework that generates client-specific adapters by conditioning the client information. |
| Outcome: | The proposed framework maximizes the utility of shared model parameters while minimizing divergence caused by client heterogeneity. |
Learning from Missing Relations: Contrastive Learning with Commonsense Knowledge Graphs for Commonsense Inference (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to commonsense inference lack coverage and expressive diversity of commonsensense knowledge graphs. |
| Approach: | They propose a framework that contrasts sets of semantically similar and dissimilar events . they propose 'solar' framework that can be used to learn commonsense inference . |
| Outcome: | The proposed framework outperforms the state-of-the-art commonsense transformer on commonsensense inference by 1.84% on average among 8 metrics. |
MELT: Materials-aware Continued Pre-training for Language Model Adaptation to Materials Science (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods focused on constructing domain-specific corpus focus on a limited and scarce nature of datasets in materials science poses significant challenges for developing models that generalize well across a broad range of materials entities. |
| Approach: | They propose a method to adapt pre-trained language models for materials science by continuously pre-training them on a materials science corpus. |
| Outcome: | The proposed method is able to adapt pre-trained language models for materials science tasks. |
Efficient Pre-training of Masked Language Model via Concept-based Curriculum Masking (2022.emnlp-main)
Copied to clipboard
| Challenge: | Masked language modeling (MLM) has been widely used for pre-training effective bidirectional representations but comes at a substantial training cost. |
| Approach: | They propose a concept-based curriculum masking method that evaluates the MLM difficulty of each token based on a carefully-designed linguistic difficulty criterion. |
| Outcome: | The proposed method significantly improves pre-training efficiency with the original BERT model at half the training cost. |