Papers by Juhyeong Park
SEED: Semantic Knowledge Transfer for Language Model Adaptation to Materials Science (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods to adapt pre-trained language models to materials science rely on frequency information from limited downstream datasets. |
| Approach: | They propose a vocabulary expansion method to adapt pre-trained language models to materials science by incorporating latent materials knowledge of lightweight embeddings into PLMs. |
| Outcome: | The proposed method mitigates the limitations of existing adaptation methods and can be used in materials science. |
KOMBO: Korean Character Representations Based on the Combination Rules of Subcharacters (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing pre-trained language models for Korean have overlooked character representation principles . a new framework for Korean PLMs uses the invention principles of Hangeul to represent letters . |
| Approach: | They propose a framework that uses Hangeul's invention principles to represent character in Korean . their method outperforms the state-of-the-art Korean PLM by 2.11% in five tasks . |
| Outcome: | The proposed method outperforms the state-of-the-art Korean PLM by 2.11% in five Korean natural language understanding tasks. |
Improving Bias Mitigation through Bias Experts in Natural Language Understanding (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to mitigate the detrimental effect of bias on the network include debiasing methods that down-weight the biased examples identified by an auxiliary model, which is trained with explicit bias labels. |
| Approach: | They propose a framework that introduces binary classifiers between the auxiliary model and main model, coined bias experts, to reduce the detrimental effect of bias on the network. |
| Outcome: | The proposed approach outperforms the state-of-the-art on various datasets while achieving high performance on in-distribution data. |
SCRIPT: A Subcharacter Compositional Representation Injection Module for Korean Pre-Trained Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Korean uses a featural writing system in which each character is composed of subcharacter units known as Jamo. |
| Approach: | They propose a model-agnostic module that injects subcharacter compositional knowledge into Korean language models. |
| Outcome: | a new module improves embeddings of Korean subwords with structural granularity . the module improve grammatical regularities and semantic cohesive variations . |