Papers by Jin-Hwa Kim
Text2Chart31: Instruction Tuning for Chart Generation with Automatic Feedback (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets do not cover full range of chart types, such as 3D, volumetric, and gridded charts. |
| Approach: | They propose a hierarchical pipeline and a new dataset for chart generation that leverages the relationships within rich datasets. |
| Outcome: | The proposed method outperforms open-source models and is comparable to state-of-the-art proprietary models in data visualization tasks. |
Query-Efficient Black-Box Red Teaming via Bayesian Optimization (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for generating test cases and querying fail to be query-efficient . generative models can be used for open-domain dialogue, prompt continuation, text-to-image generation . |
| Approach: | They propose a query-efficient method that iteratively finds diverse positive test cases leading to model failures by utilizing user input and past evaluations. |
| Outcome: | The proposed method finds a significantly larger number of diverse positive test cases under limited query budget than baseline methods. |
Modal-specific Pseudo Query Generation for Video Corpus Moment Retrieval (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown promising results in video corpus moment retrieval . however, they relied on the expensive query annotations for the VCMR . |
| Approach: | They propose a self-supervised learning framework to localize video corpus moment without annotations. |
| Outcome: | The proposed framework can localize the video corpus moment without any explicit annotation on TVR dataset. |
CoDraw: Collaborative Drawing as a Testbed for Grounded Goal-driven Communication (P19-1)
Copied to clipboard
Jin-Hwa Kim, Nikita Kitaev, Xinlei Chen, Marcus Rohrbach, Byoung-Tak Zhang, Yuandong Tian, Dhruv Batra, Devi Parikh
| Challenge: | a goal-driven collaborative drawing task combines language, perception, and actions in a partially observable environment . et al., 1990: 138K messages exchanged between human players. |
| Approach: | They propose a goal-driven collaborative task that combines language, perception, and action . they collect a clip art dataset and use it to build an image-drawing game between two agents . |
| Outcome: | The proposed task integrates language, perception, and action in a virtual world . it is based on a dataset of 10K dialogs and 138K messages exchanged between humans . |
TimeChara: Evaluating Point-in-Time Character Hallucination of Role-Playing Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can be used to simulate human behaviors, but point-in-time role-playing is a key component of fandom role-players. |
| Approach: | They propose a benchmark to evaluate point-in-time character hallucination in role-playing LLMs. |
| Outcome: | The proposed method reduces point-in-time character hallucinations effectively by decomposing reasoning steps and using narrative experts. |
AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models (2022.findings-emnlp)
Copied to clipboard
Se Jung Kwon, Jeonghoon Kim, Jeongin Bae, Kang Min Yoo, Jin-Hwa Kim, Baeseong Park, Byeongwook Kim, Jung-Woo Ha, Nako Sung, Dongsoo Lee
| Challenge: | Existing approaches to improve inference efficiency by accelerating model fine-tuning have not been thoroughly explored. |
| Approach: | They propose to combine parameter-efficient adaptation and model compression to accelerate model . they propose to freeze binary parameters and scale scaling factors for target tasks . |
| Outcome: | The proposed algorithm achieves >10x compression ratio under 4-bit quantization and >1,000x reduction in trainable parameters. |
Reasoning Visual Dialog with Sparse Graph Learning and Knowledge Transfer (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Visual dialog is a task of answering questions grounded in an image using dialog history as context. |
| Approach: | They propose a Sparse Graph Learning method to formulate visual dialog as a graph structure learning task. |
| Outcome: | The proposed model outperforms the state-of-the-art models on the VisDial v1.0 dataset. |