Papers by Soyoung Park
Two Examples are Better than One: Context Regularization for Gradient-based Prompt Tuning (2023.findings-acl)
Copied to clipboard
| Challenge: | Prompting has gained tremendous attention as an efficient method for the adaptation of large-scale language models. |
| Approach: | They propose a regularization method that guides a prompt to produce a task context properly. |
| Outcome: | The proposed method improves prediction performance in a zero-shot in-context learning setting without demonstration examples for in-constitu learning. |
Unsupervised Neural Machine Translation for Low-Resource Domains via Meta-Learning (2021.acl-long)
Copied to clipboard
| Challenge: | Unsupervised machine translation suffers from data-scarce domains, authors report . a meta-learning algorithm trains the model to adapt to another domain by utilizing only a small amount of training data. |
| Approach: | They propose a meta-learning algorithm that trains the model to adapt to another domain . their model surpasses a transfer learning-based approach by up to 2-3 BLEU scores . |
| Outcome: | The proposed algorithm outperforms a transfer learning-based approach by 2-3 BLEU scores . the proposed model outperformed previous models in the domain of unsupervised machine translation . |
What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers (2021.emnlp-main)
Copied to clipboard
Boseop Kim, HyoungSeok Kim, Sang-Woo Lee, Gichang Lee, Donghyun Kwak, Jeon Dong Hyeon, Sunghyun Park, Sungju Kim, Seonhoon Kim, Dongpil Seo, Heungsub Lee, Minyoung Jeong, Sungjae Lee, Minsub Kim, Suk Hyun Ko, Seokhun Kim, Taeyong Park, Jinuk Kim, Soyoung Kang, Na-Hyeon Ryu, Kang Min Yoo, Minsuk Chang, Soobin Suh, Sookyo In, Jinseong Park, Kyungduk Kim, Hiun Kim, Jisu Jeong, Yong Goo Yeo, Donghoon Ham, Dongju Park, Min Young Lee, Jaewook Kang, Inho Kang, Jung-Woo Ha, Woomyoung Park, Nako Sung
| Challenge: | GPT-3 has been used to train large-scale language models on hundreds of billion scale data. |
| Approach: | They propose a Korean variant of GPT-3 that uses Korean tokens to train in-context models. |
| Outcome: | The proposed method shows state-of-the-art zero-shot and few-shot learning on downstream tasks in Korean. |
Cross-Lingual Suicidal-Oriented Word Embedding toward Suicide Prevention (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing suicide dictionaries for other languages have been limited to Korean . a model that uses social media data to identify whether a post includes suicidal ideation is useful . |
| Approach: | They propose a model that uses existing suicide dictionaries for Korean to predict suicidal ideation . they use the existing dictionary for English and Chinese to translate a post into English and then use the separate suicide-oriented embeddings for English. |
| Outcome: | The proposed model can detect whether a given social media post includes suicidal ideation in Korean . it uses existing suicide dictionaries for other languages to translate the post into English and Chinese, and then embeds the suicide-oriented embeddings for English and China. |
Keep Me Updated! Memory Management in Long-term Conversations (2022.findings-emnlp)
Copied to clipboard
Sanghwan Bae, Donghyun Kwak, Soyoung Kang, Min Young Lee, Sungdong Kim, Yuin Jeong, Hyeri Kim, Sang-Woo Lee, Woomyoung Park, Nako Sung
| Challenge: | Existing studies do not deal with cases where memorized information is outdated, which may cause confusion in later conversations. |
| Approach: | They propose a task where bots keep track of and bring up the latest information about users while conversing through multiple sessions. |
| Outcome: | The proposed method outperforms baselines that leave the stored memory unchanged in terms of engagingness and humanness, and a larger performance gap in the later sessions. |
Towards standardizing Korean Grammatical Error Correction: Datasets and Annotation (2023.acl-long)
Copied to clipboard
| Challenge: | Despite the growing number of Korean learners, little research has been conducted on Korean grammatical error correction (GEC) despite the difficulties of the Korean language, there is no evaluation benchmark for Korean GEC. |
| Approach: | They propose to use Korean grammar error correction datasets to train a machine learning model that can automatically annotate Korean errors from parallel corpora. |
| Outcome: | The proposed model outperforms the currently used statistical Korean GEC system on a wider range of error types. |
SSMix: Saliency-Based Span Mixup for Text Classification (2021.findings-acl)
Copied to clipboard
| Challenge: | SSMix synthesizes a sentence while preserving the locality of two original texts by span-based mixing and keeping more tokens related to the prediction relying on saliency information. |
| Approach: | They propose a new method where the operation is performed on input text rather than on hidden vectors like previous approaches. |
| Outcome: | The proposed method outperforms hidden-level mixup methods on a wide range of text classification benchmarks including textual entailment, sentiment classification, and questiontype classification. |
Building a Role Specified Open-Domain Dialogue System Leveraging Large-Scale Language Models (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent large-scale language models have produced human-like responses in open-domain dialogue systems. |
| Approach: | They propose a framework for imposing roles on open-domain dialogue systems . they use few-shot learning to build a Korean dialogue dataset from scratch . |
| Outcome: | The proposed framework meets role specifications while maintaining conversational abilities. |