Papers by Seonhoon Kim
Korean Language Modeling via Syntactic Guide (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing research on pre-trained language models focuses on widely-used languages . however, not every language can benefit from such models due to computational resources . |
| Approach: | They propose to build a pre-trained language model that understands the linguistic phenomena in the target language with low resources. |
| Outcome: | The proposed model improves the performance of Korean language understanding tasks. |
Textbook Question Answering with Multi-modal Context Graph Understanding and Self-supervised Open-set Comprehension (P19-1)
Copied to clipboard
| Challenge: | In a decade, question answering (QA) has been one of the most promising achievements in the field of natural language processing (NLP). |
| Approach: | They propose a new module f-GCN based on graph convolutional networks (GCN) to extract knowledge features from multi-modal contexts in complex input data. |
| Outcome: | The proposed model outperforms state-of-the-art methods on the textbook question answering task and on the visual features of the dataset. |
LM-BFF-MS: Improving Few-Shot Fine-tuning of Language Models based on Multiple Soft Demonstration Memory (2022.acl-short)
Copied to clipboard
| Challenge: | LM-BFF (CITATION) achieves significant few-shot performance by using auto-generated prompts and adding demonstrations similar to an input example. |
| Approach: | They propose to use auto-generated prompts and add demonstrations to LM-BFF to improve few-shot fine-tuning of language models with multiple soft demonstrations. |
| Outcome: | The proposed method improves few-shot fine-tuning on eight NLP tasks. |
What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers (2021.emnlp-main)
Copied to clipboard
Boseop Kim, HyoungSeok Kim, Sang-Woo Lee, Gichang Lee, Donghyun Kwak, Jeon Dong Hyeon, Sunghyun Park, Sungju Kim, Seonhoon Kim, Dongpil Seo, Heungsub Lee, Minyoung Jeong, Sungjae Lee, Minsub Kim, Suk Hyun Ko, Seokhun Kim, Taeyong Park, Jinuk Kim, Soyoung Kang, Na-Hyeon Ryu, Kang Min Yoo, Minsuk Chang, Soobin Suh, Sookyo In, Jinseong Park, Kyungduk Kim, Hiun Kim, Jisu Jeong, Yong Goo Yeo, Donghoon Ham, Dongju Park, Min Young Lee, Jaewook Kang, Inho Kang, Jung-Woo Ha, Woomyoung Park, Nako Sung
| Challenge: | GPT-3 has been used to train large-scale language models on hundreds of billion scale data. |
| Approach: | They propose a Korean variant of GPT-3 that uses Korean tokens to train in-context models. |
| Outcome: | The proposed method shows state-of-the-art zero-shot and few-shot learning on downstream tasks in Korean. |
SISER: Semantic-Infused Selective Graph Reasoning for Fact Verification (2022.coling-1)
Copied to clipboard
| Challenge: | Existing graph-based methods for fact verification use semantic graphs, which are based on evidence sentences. |
| Approach: | They propose to use semantic-level graph reasoning to inject its reasoning-enhanced representation into other graph-based and sequence-based reasoning methods. |
| Outcome: | The proposed method outperforms the previous graph-based methods and achieves state-of-the-art performance on a large-scale dataset for Fact Extraction and VERification (FEVER). |