Papers by Jaesik Kim
Extending CLIP’s Image-Text Alignment to Referring Image Segmentation (2024.naacl-long)
Copied to clipboard
| Challenge: | Referring Image Segmentation (RIS) is a cross-modal task that aims to segment an instance described by a natural language expression. |
| Approach: | They propose a framework that leverages the cross-modal nature of CLIP for RIS by leveraging image-text alignment knowledge in CLIP's image-embedding space. |
| Outcome: | The proposed framework outperforms CLIP-based methods on all three major RIS benchmarks and outperformed previous CLIP methods. |
Incomplete Prompt Jailbreaks in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly released as open-weight models with safeguards against harmful requests. |
| Approach: | They formalize incomplete prompt jailbreaks as incomplete prompts elicit harmful continuations . they identify two functional neurons that delay refusal until sentence termination . |
| Outcome: | The proposed model fails to generalize across content domains and attractor types . the proposed model can be used to perform more precise and robust IPJ defenses . |
When Format Changes Meaning: Investigating Semantic Inconsistency of Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models are vulnerable to semantic inconsistency, a study finds . minor formatting variations result in divergent predictions for semantically equivalent inputs. |
| Approach: | They evaluate LLMs for semantic inconsistency and find they remain vulnerable . they propose to use mechanistic analysis to develop models that improve their reliability . |
| Outcome: | The proposed model is vulnerable to semantic inconsistency, the authors show . their model is brittle even in state-of-the-art models, they say . |
Improving Unsupervised Out-of-domain detection through Pseudo Labeling and Learning (2023.findings-eacl)
Copied to clipboard
| Challenge: | Unsupervised OOD detection is a task aimed at discriminating whether given samples are from the in-domain (IND) . previous studies adopted the one-class classification approach, assuming that the training samples come from a single domain. |
| Approach: | They propose a framework that leverages latent categorical information to improve representation learning for textual OOD detection. |
| Outcome: | The proposed framework significantly outperforms baseline models on three datasets. |
K-MetBench: A Multi-Dimensional Benchmark for Fine-Grained Evaluation of Expert Reasoning, Locality, and Multimodality in Meteorology (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation frameworks for large language models for domain specific tasks are coarse and do not provide a multidimensional evaluation of a model's ability to interpret domain specific data. |
| Approach: | They propose a diagnostic benchmark grounded in national qualification exams that exposes critical gaps across four dimensions: expert visual reasoning of charts, logical validity via expert-verified rationales, Korean-specific geo-cultural comprehension, and fine-grained domain analysis. |
| Outcome: | The proposed model outperforms global models in local contexts, demonstrating that parameter scaling alone cannot resolve cultural dependencies. |