Papers by Geewook Kim
Evaluating Multimodal Generative AI with Korean Educational Standards (2025.naacl-short)
Copied to clipboard
| Challenge: | Current benchmarks focus on English, overlooking the linguistic diversity worldwide and offering limited insights into low-resource languages like Korean. |
| Approach: | They propose to use Korean national educational tests to evaluate AI systems using a benchmark dataset. |
| Outcome: | The proposed benchmarks evaluate models in less-explored languages and open-source code and dataset builder will be fully open-sourced. |
Segmentation-free compositional n-gram embedding (N19-1)
Copied to clipboard
| Challenge: | Existing word embedding models depend on word segmentation, but this method is difficult when corpora written in noisy or unsegmented languages. |
| Approach: | They propose a new method that models words, phrases and sentences seamlessly without word segmentation. |
| Outcome: | The proposed method is very effective for noisy corpora written in unsegmented languages such as Chinese and Japanese. |
Cost-effective End-to-end Information Extraction for Semi-structured Document Images (2021.emnlp-main)
Copied to clipboard
| Challenge: | a real-world information extraction system for semi-structured document images often involves a long pipeline of multiple modules, which can lead to unstable performance if not designed carefully. |
| Approach: | They propose to use a sequence generation task to build an end-to-end IE system . they propose to combine three manually engineered modules with one data-driven module . |
| Outcome: | The proposed system can be easily replaced and deployed in large-scale production. |
Prometheus-Vision: Vision-Language Model as a Judge for Fine-Grained Evaluation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing metrics for long-form text outputs are prone to biases and scaling up is expensive. |
| Approach: | They propose to evaluate VLMs with VLM feedback dataset . they use 15K customized score rubrics to train Prometheus-Vision . |
| Outcome: | The proposed model shows highest correlation with human evaluators and GPT-4V among open-source models. |
MMRefine: Unveiling the Obstacles to Robust Refinement in Multimodal Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances have enabled MLLMs to tackle complex challenges such as mathematical reasoning and multimodal understanding. |
| Approach: | They propose a multimodal refinement benchmark to evaluate the refinement capabilities of Multimodal Large Language Models (MLLMs) the benchmark categorizes errors into six error types to highlight areas for improvement in effective reasoning enhancement. |
| Outcome: | The proposed framework evaluates the refinement capabilities of multimodal large language models across six scenarios. |
Visually-Situated Natural Language Understanding with Contrastive Reading Model and Frozen Large Language Models (2023.emnlp-main)
Copied to clipboard
Geewook Kim, Hodong Lee, Daehee Kim, Haeji Jung, Sanghee Park, Yoonsik Kim, Sangdoo Yun, Taeho Kil, Bado Lee, Seunghyun Park
| Challenge: | Recent advances in Large Language Models (LLMs) have stimulated a surge of research aimed at extending their applications to the visual domain. |
| Approach: | They propose a novel neural architecture to enhance language-image understanding capability of LLMs by capturing intricate details that are often overlooked in existing methods. |
| Outcome: | The proposed model can achieve better comprehension of language information in visual contexts within images. |
Scale down Transformer by Grouping Features for a Lightweight Character-level Language Model (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to character-level language modeling have suffered from high learning complexity caused by inherently long character sequences. |
| Approach: | They propose a method that efficiently reduces the computational cost and parameter size of Transformer by splitting feature space into multiple groups, factorizing the calculation paths, and reducing computations for the group interaction. |
| Outcome: | The proposed model reduces the computational cost and parameter size of Transformer on two benchmark tasks, enwik8 and text8, and it performs well. |
Instruction Tuning with and without Context: Behavioral Shifts and Downstream Impact (2026.eacl-long)
Copied to clipboard
Hyunji Lee, Seunghyun Yoon, Yunjae Won, Hanseok Oh, Geewook Kim, Trung Bui, Franck Dernoncourt, Elias Stengel-Eskin, Mohit Bansal, Minjoon Seo
| Challenge: | Prior work on instruction tuning datasets combined these data types without examining their distinct effects. |
| Approach: | They investigate how training LLMs with or without context affects model behavior and performance . they find that using context-augmented data as the backbone for vision-language models reduces hallucination . |
| Outcome: | The proposed training with context-augmented data reduces hallucination and improves grounding in the visual domain. |
On Efficient Language and Vision Assistants for Visually-Situated Natural Language Understanding: What Matters in Reading and Reasoning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in language and vision assistants have showcased impressive capabilities but suffer from a lack of transparency, limiting broader research and reproducibility. |
| Approach: | They propose to redefine the design of vision-language models by identifying key components and creating efficient models with constrained inference costs. |
| Outcome: | The proposed models achieve significant improvements in inference throughput while maintaining high performance. |