Papers by Jung-Woo Ha
NL2pSQL: Generating Pseudo-SQL Queries from Under-Specified Natural Language Questions (D19-1)
Copied to clipboard
| Challenge: | Existing studies focus on generating SQL codes from natural language questions . however, questions cover more diverse tasks including table manipulation or performance issues . |
| Approach: | They propose a task to generate pSQL codes from natural language questions . they define two new metrics suitable for the task, Canonical-BLEU and SQL-BLUE . |
| Outcome: | The proposed task generates well-formed queries on under-specified database issues. |
Continuous Decomposition of Granularity for Neural Paraphrase Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Prior work has shown that decomposing sentences at different levels of granularity has improved paragraph generation. |
| Approach: | They propose a model for continuous decomposing granularity for neural paraphrase generation that incorporates granules into attention. |
| Outcome: | The proposed model outperforms baseline models on Quora question pairs and Twitter URLs on two benchmarks. |
What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers (2021.emnlp-main)
Copied to clipboard
Boseop Kim, HyoungSeok Kim, Sang-Woo Lee, Gichang Lee, Donghyun Kwak, Jeon Dong Hyeon, Sunghyun Park, Sungju Kim, Seonhoon Kim, Dongpil Seo, Heungsub Lee, Minyoung Jeong, Sungjae Lee, Minsub Kim, Suk Hyun Ko, Seokhun Kim, Taeyong Park, Jinuk Kim, Soyoung Kang, Na-Hyeon Ryu, Kang Min Yoo, Minsuk Chang, Soobin Suh, Sookyo In, Jinseong Park, Kyungduk Kim, Hiun Kim, Jisu Jeong, Yong Goo Yeo, Donghoon Ham, Dongju Park, Min Young Lee, Jaewook Kang, Inho Kang, Jung-Woo Ha, Woomyoung Park, Nako Sung
| Challenge: | GPT-3 has been used to train large-scale language models on hundreds of billion scale data. |
| Approach: | They propose a Korean variant of GPT-3 that uses Korean tokens to train in-context models. |
| Outcome: | The proposed method shows state-of-the-art zero-shot and few-shot learning on downstream tasks in Korean. |
Query-Efficient Black-Box Red Teaming via Bayesian Optimization (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for generating test cases and querying fail to be query-efficient . generative models can be used for open-domain dialogue, prompt continuation, text-to-image generation . |
| Approach: | They propose a query-efficient method that iteratively finds diverse positive test cases leading to model failures by utilizing user input and past evaluations. |
| Outcome: | The proposed method finds a significantly larger number of diverse positive test cases under limited query budget than baseline methods. |
Pivotal Role of Language Modeling in Recommender Systems: Enriching Task-specific and Task-agnostic Representation Learning (2023.acl-long)
Copied to clipboard
Kyuyong Shin, Hanock Kwak, Wonjae Kim, Jisu Jeong, Seungjae Jung, Kyungmin Kim, Jung-Woo Ha, Sang-Woo Lee
| Challenge: | Recent studies have proposed unified user modeling frameworks that leverage user behavior data from various applications. |
| Approach: | They propose to use user behavior sequences as plain text to represent rich information in any domain or system without losing generality. |
| Outcome: | The proposed frameworks achieve excellent results on diverse recommendation tasks and can be used on unseen domains and services. |
SQuARe: A Large-Scale Dataset of Sensitive Questions and Acceptable Responses Created through Human-Machine Collaboration (2023.acl-long)
Copied to clipboard
Hwaran Lee, Seokhee Hong, Joonsuk Park, Takyoung Kim, Meeyoung Cha, Yejin Choi, Byoungpil Kim, Gunhee Kim, Eun-Ju Lee, Yong Lim, Alice Oh, Sangchul Park, Jung-Woo Ha
| Challenge: | Existing studies focus on coping with social harms that large language models pose . however, discussions on sensitive issues can become toxic even if the users are well-intentioned. |
| Approach: | They propose to use Korean dataset to test whether LLMs can generate offensive content and propagate prejudices. |
| Outcome: | The proposed dataset shows that acceptable response generation improves for HyperCLOVA and GPT-3. |
Context-Aware Answer Extraction in Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Extractive QA models have shown promising performance in predicting the correct answer to a given question. |
| Approach: | They propose a BLANC-based context prediction task that learns the context prediction tasks. |
| Outcome: | The proposed model outperforms the state-of-the-art models on reading comprehension and hotpotQA. |
On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Model (2022.naacl-main)
Copied to clipboard
Seongjin Shin, Sang-Woo Lee, Hwijeen Ahn, Sungdong Kim, HyoungSeok Kim, Boseop Kim, Kyunghyun Cho, Gichang Lee, Woomyoung Park, Jung-Woo Ha, Nako Sung
| Challenge: | Recent studies on large-scale in-context language models have reported successful in-const zero- and few-shot learning ability. |
| Approach: | They investigate the effects of the pretraining corpus on in-context learning in a Korean-centric model. |
| Outcome: | The study shows that pretraining corpus size does not determine in-context learning ability . the findings suggest that in-constext learning is not always competitive . |
AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models (2022.findings-emnlp)
Copied to clipboard
Se Jung Kwon, Jeonghoon Kim, Jeongin Bae, Kang Min Yoo, Jin-Hwa Kim, Baeseong Park, Byeongwook Kim, Jung-Woo Ha, Nako Sung, Dongsoo Lee
| Challenge: | Existing approaches to improve inference efficiency by accelerating model fine-tuning have not been thoroughly explored. |
| Approach: | They propose to combine parameter-efficient adaptation and model compression to accelerate model . they propose to freeze binary parameters and scale scaling factors for target tasks . |
| Outcome: | The proposed algorithm achieves >10x compression ratio under 4-bit quantization and >1,000x reduction in trainable parameters. |
Weakly Supervised Pre-Training for Multi-Hop Retriever (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for weakly supervised multi-hop pretraining require costly human annotation. |
| Approach: | They propose a method for weakly supervised multi-hop retriever pretraining without human efforts by generating vector representations of complex questions and subquestion as weak supervision for pre-training. |
| Outcome: | The proposed method is effective and robust on limited data and computational resources. |
Two-Step Question Retrieval for Open-Domain QA (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing question retrieval models have shown a significant increase in inference speed but at the cost of lower QA performance compared to the retriever-reader pipeline. |
| Approach: | They propose a two-step question retrieval model with distant supervision to improve inference speed. |
| Outcome: | The proposed model significantly increases the performance of existing question retrieval models with a negligible loss on inference speed. |