Papers by Jiwan Chung
EgoSpeak: Learning When to Speak for Egocentric Conversational Agents in the Wild (2025.findings-naacl)
Copied to clipboard
Junhyeok Kim, Min Soo Kim, Jiwan Chung, Jungbin Cho, Jisoo Kim, Sungwoong Kim, Gyeongbo Sim, Youngjae Yu
| Challenge: | EgoSpeak predicts when an agent should begin speaking based on egocentric streaming video. |
| Approach: | They propose a framework for real-time speech initiation prediction in egocentric streaming video by modeling the conversation from the camera wearer's first-person perspective. |
| Outcome: | The proposed framework outperforms random and silence-based baselines in real time and highlights the importance of multimodal input and context length in effectively deciding when to speak. |
Can visual language models resolve textual ambiguity with visual cues? Let visual puns tell you! (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing models lack this active understanding capacity, limiting their applicability in real-world scenarios. |
| Approach: | They propose a benchmark to assess the impact of multimodal inputs on lexical ambiguities. |
| Outcome: | The proposed benchmark assesses the impact of multimodal inputs on lexical ambiguities. |
Are Any-to-Any Models More Consistent Across Modality Transfers Than Specialists? (2025.acl-long)
Copied to clipboard
| Challenge: | equivariance evaluations uncover weak but observable consistency through structured analyses of the intermediate latent space enabled by multiple editing operations. |
| Approach: | They use a dataset of 1,000 images paired with captions, editing instructions, and Q&A pairs to evaluate cross-modal transfers rigorously. |
| Outcome: | The proposed models do not consistently demonstrate greater cross-modal consistency than specialized models in pointwise evaluations such as cyclic consistency. |
Selective Vision is the Challenge for Visual Reasoning: A Benchmark for Visual Argument Understanding (2024.emnlp-main)
Copied to clipboard
| Challenge: | Visual arguments rely on images to persuade viewers to do or believe something . |
| Approach: | They propose three tasks for evaluating visual argument understanding . they use visual premises, commonsense premises and reasoning trees to analyze visual arguments . |
| Outcome: | The proposed tasks evaluate visual argument understanding using a dataset of 1,611 images annotated with 5,112 visual premises (with regions), 5,574 commonsense premises, and reasoning trees connecting them into structured arguments. |
VLIS: Unimodal Language Models Guide Multimodal Language Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing vision-language models face challenges in tasks that require complex linguistic understanding. |
| Approach: | They propose a framework that combines visual conditioning and linguistic understanding of unimodal text-only language models without further training to improve vision-language models. |
| Outcome: | The proposed framework improves vision-language models on diverse tasks including commonsense understanding and complex text generation. |
Reading Books is Great, But Not if You Are Driving! Visually Grounded Reasoning about Defeasible Commonsense Norms (2023.emnlp-main)
Copied to clipboard
Seungju Han, Junhyeok Kim, Jack Hessel, Liwei Jiang, Jiwan Chung, Yejin Son, Yejin Choi, Youngjae Yu
| Challenge: | NormLens is a visual-grounded framework for understanding commonsense norms . state-of-the-art models are not well-aligned with human annotation, we show . |
| Approach: | They propose a visual-grounded framework to study commonsense norms by NormLens . they find that models are not well-aligned with human annotation . |
| Outcome: | The proposed model judgments and explanations are not well-aligned with human annotations. |
Speaking Beyond Language: A Large-Scale Multimodal Dataset for Learning Nonverbal Cues from Video-Grounded Dialogues (2025.acl-long)
Copied to clipboard
Youngmin Kim, Jiwan Chung, Jisoo Kim, Sunghyun Lee, Sangkyu Lee, Junhyeok Kim, Cheoljong Yang, Youngjae Yu
| Challenge: | Existing large language models fail to incorporate nonverbal elements into conversational experiences. |
| Approach: | They propose a multimodal language model that generates nonverbal cues alongside text . their dataset is annotated with time-aligned text, facial expressions, and body language . |
| Outcome: | The proposed model generates nonverbal languages and text, corresponding to conversational input. |
GuideDog: A Real-World Egocentric Multimodal Dataset for Blind and Low-Vision Accessibility-Aware Guidance (2026.acl-long)
Copied to clipboard
Junhyeok Kim, Jaewoo Park, Junhee Park, Sangeyl Lee, Jiwan Chung, Jisung Kim, Ji Hoon Joung, Youngjae Yu
| Challenge: | Recent advances in multimodal large language models (MLLMs) offer new opportunities for higher-level scene understanding, but they require labor-intensive, expert annotation. |
| Approach: | They propose a dataset that combines 2K human-verified images with 22K image-description pairs to provide a more accurate representation of pedestrian scenes. |
| Outcome: | The proposed dataset improves scalability while maintaining quality. |
Language Models as Compilers: Simulating Pseudocode Execution Improves Algorithmic Reasoning in Language Models (2024.emnlp-main)
Copied to clipboard
Hyungjoo Chae, Yeonghyeon Kim, Seungone Kim, Kai Ong, Beong-woo Kwak, Moohyeon Kim, Sunghwan Kim, Taeyoon Kwon, Jiwan Chung, Youngjae Yu, Jinyoung Yeo
| Challenge: | Prior work has used LLMs to generate programming language and applied external compilers for such tasks. |
| Approach: | They propose a framework that expresses task-level logic with pseudocode and tailors it to each instance and simulates execution of it. |
| Outcome: | The proposed framework outperforms baselines in diverse reasoning tasks. |
Do LLMs Have Distinct and Consistent Personality? TRAIT: Personality Testset designed for LLMs with Psychometrics (2025.findings-naacl)
Copied to clipboard
Seungbeen Lee, Seungwon Lim, Seungju Han, Giyeong Oh, Hyungjoo Chae, Jiwan Chung, Minju Kim, Beong-woo Kwak, Yeonsoo Lee, Dongha Lee, Jinyoung Yeo, Youngjae Yu
| Challenge: | Recent advances in Large Language Models (LLMs) have led to their adaptation as conversational agents. |
| Approach: | They propose a new benchmark that uses 8K multi-choice questions to assess the personality of Large Language Models. |
| Outcome: | The proposed personality test outperforms existing personality tests for LLMs in reliability and validity. |
VisEscape: A Benchmark for Evaluating Exploration-driven Decision-making in Virtual Escape Rooms (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on embodied agents have addressed the importance of exploration in environments where tasks and solutions are not predefined. |
| Approach: | They propose a virtual escape room that evaluates AI models in a dynamic environment . they propose to integrate memory management and reasoning into the simulation . |
| Outcome: | The proposed model improves in dynamic and exploration-driven environments by integrating memory management and reasoning. |