Papers by Soyoung Kang
Restoring and Mining the Records of the Joseon Dynasty via Neural Language Modeling and Machine Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | voluminous historical records are difficult to fully utilize since they are written in ancient languages and some parts are damaged over time. |
| Approach: | They propose a multi-task learning approach to restore and translate historical documents using a self-attention mechanism. |
| Outcome: | The proposed approach improves the accuracy of the translation task over baselines without multi-task learning. |
What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers (2021.emnlp-main)
Copied to clipboard
Boseop Kim, HyoungSeok Kim, Sang-Woo Lee, Gichang Lee, Donghyun Kwak, Jeon Dong Hyeon, Sunghyun Park, Sungju Kim, Seonhoon Kim, Dongpil Seo, Heungsub Lee, Minyoung Jeong, Sungjae Lee, Minsub Kim, Suk Hyun Ko, Seokhun Kim, Taeyong Park, Jinuk Kim, Soyoung Kang, Na-Hyeon Ryu, Kang Min Yoo, Minsuk Chang, Soobin Suh, Sookyo In, Jinseong Park, Kyungduk Kim, Hiun Kim, Jisu Jeong, Yong Goo Yeo, Donghoon Ham, Dongju Park, Min Young Lee, Jaewook Kang, Inho Kang, Jung-Woo Ha, Woomyoung Park, Nako Sung
| Challenge: | GPT-3 has been used to train large-scale language models on hundreds of billion scale data. |
| Approach: | They propose a Korean variant of GPT-3 that uses Korean tokens to train in-context models. |
| Outcome: | The proposed method shows state-of-the-art zero-shot and few-shot learning on downstream tasks in Korean. |
Cross-Lingual Suicidal-Oriented Word Embedding toward Suicide Prevention (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing suicide dictionaries for other languages have been limited to Korean . a model that uses social media data to identify whether a post includes suicidal ideation is useful . |
| Approach: | They propose a model that uses existing suicide dictionaries for Korean to predict suicidal ideation . they use the existing dictionary for English and Chinese to translate a post into English and then use the separate suicide-oriented embeddings for English. |
| Outcome: | The proposed model can detect whether a given social media post includes suicidal ideation in Korean . it uses existing suicide dictionaries for other languages to translate the post into English and Chinese, and then embeds the suicide-oriented embeddings for English and China. |
Keep Me Updated! Memory Management in Long-term Conversations (2022.findings-emnlp)
Copied to clipboard
Sanghwan Bae, Donghyun Kwak, Soyoung Kang, Min Young Lee, Sungdong Kim, Yuin Jeong, Hyeri Kim, Sang-Woo Lee, Woomyoung Park, Nako Sung
| Challenge: | Existing studies do not deal with cases where memorized information is outdated, which may cause confusion in later conversations. |
| Approach: | They propose a task where bots keep track of and bring up the latest information about users while conversing through multiple sessions. |
| Outcome: | The proposed method outperforms baselines that leave the stored memory unchanged in terms of engagingness and humanness, and a larger performance gap in the later sessions. |
Building a Role Specified Open-Domain Dialogue System Leveraging Large-Scale Language Models (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent large-scale language models have produced human-like responses in open-domain dialogue systems. |
| Approach: | They propose a framework for imposing roles on open-domain dialogue systems . they use few-shot learning to build a Korean dialogue dataset from scratch . |
| Outcome: | The proposed framework meets role specifications while maintaining conversational abilities. |
Aligning Large Language Models through Synthetic Feedback (2023.emnlp-main)
Copied to clipboard
| Challenge: | Currently, alignment learning requires significant human demonstrations and feedback from proprietary LLMs such as ChatGPT. |
| Approach: | They propose a framework that uses synthetic feedback to align large language models to human values without extensive human annotations and proprietary LLMs. |
| Outcome: | The proposed model outperforms open-source models on human-annotated demonstrations in alignment benchmarks. |