Papers by Hyunwoo Kim
Deal, or no deal (or who knows)? Forecasting Uncertainty in Conversations using Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Effective interlocutors account for the uncertain goals, beliefs, and emotions of others. |
| Approach: | They propose to calibrate language models to better represent outcome uncertainty . they propose to use two methods to calibrated small open-source models . |
| Outcome: | The proposed fine-tuning strategies can calibrate smaller open-source models to beat pre-trained models 10x their size. |
SLM as Guardian: Pioneering AI Safety with Small Language Model (2024.emnlp-industry)
Copied to clipboard
Ohjoon Kwon, Donghyeon Jeon, Nayoung Choi, Gyu-Hwung Cho, Hwiyeol Jo, Changbong Kim, Hyunwoo Lee, Inho Kang, Sun Kim, Taiwoo Park
| Challenge: | Prior safety research on large language models focused on aligning them to safety requirements, but internalizing such safeguard features into larger models brought challenges of higher training cost and unintended degradation of helpfulness. |
| Approach: | They propose a multi-task learning mechanism that integrates harmful query detection and safeguard response into a single model. |
| Outcome: | The proposed approach outperforms the publicly available LLMs in harmful query detection and safeguard response generation. |
What Users Leave Unsaid: Under-Specified Queries Limit Vision-Language Models (2026.findings-acl)
Copied to clipboard
Dasol Choi, Guijin Son, Hanwool Lee, Minhyuk Kim, Hyunwoo Ko, Teabin Lim, Eungyeol Ahn, Jungwhan Kim, Seunghyeok Hong, Youngsook Song
| Challenge: | HAERAE-Vision benchmarks feature clear, explicit prompts but are often informal and underspecified . state-of-the-art models achieve under 50% on original queries, compared to GPT-5 and Gemini 2.5 Pro . |
| Approach: | They propose a benchmark of 653 real-world visual questions from Korean online communities . they find that even state-of-the-art models achieve under 50% on original queries . |
| Outcome: | HAERAE-Vision benchmarks from Korean online communities yield 1,306 query variants . state-of-the-art models achieve under 50% on original queries, compared with smaller models . authors show that query explicitation alone yields 8 to 22 point improvements . |
Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models have enabled richer social simulations . however, the role of information asymmetry in these simulations has been overlooked . |
| Approach: | They develop an evaluation framework to simulate social interactions with LLMs in different settings. |
| Outcome: | The proposed framework performs better in unrealistic, omniscient simulation settings but struggles in those with information asymmetry. |
FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluations for theory of mind (ToM) use passive narratives that lack interactivity. |
| Approach: | They propose a benchmark to stress-test ToM within information-asymmetric conversational contexts via question answering. |
| Outcome: | The proposed benchmark is challenging for state-of-the-art language models, which perform significantly worse than humans even with chain-of thought reasoning or fine-tuning. |
Will I Sound Like Me? Improving Persona Consistency in Dialogues through Pragmatic Self-Consciousness (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for improving consistency often train with additional NLI labels or attach trained extra modules to the generative agent. |
| Approach: | They propose to encode personas into dialogue embeddings and a persona-conditioned dialogue dataset to improve persona consistency. |
| Outcome: | The proposed approach can enforce dialogue agents to refrain from contradictions and improve consistency of existing models. |
Socratic-MCTS: Test-Time Visual Reasoning by Asking the Right Questions (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent research in vision-language models has centered around the possibility of equipping them with implicit long-form chain-of-thought reasoning via distillation and reinforcement learning. |
| Approach: | They propose a Monte Carlo Tree Search-inspired algorithm that injects subquestion–subanswer pairs into the model’s output stream to elicit hidden knowledge and induce long reasoning traces. |
| Outcome: | The proposed method yields a 2% improvement on MMMU-PRO, including a significant 9% gain in Liberal Arts. |
Abstractive Summarization of Reddit Posts with Multi-level Memory Networks (N19-1)
Copied to clipboard
| Challenge: | Abstractive summarization methods suffer from inferior performance compared to extractive methods. |
| Approach: | They propose a reddit TIFU dataset and a new abstractive summarization model . they use multi-level memory networks to store information from different levels of abstraction . |
| Outcome: | The proposed model outperforms state-of-the-art summarization models with multi-level memory . the proposed dataset is highly abstractive and outperformed existing models with the proposed model . |
Taxonomy and Analysis of Sensitive User Queries in Generative AI Search System (2025.findings-naacl)
Copied to clipboard
Hwiyeol Jo, Taiwoo Park, Hyunwoo Lee, Nayoung Choi, Changbong Kim, Ohjoon Kwon, Donghyeon Jeon, Eui Hyeon Lee, Kyoungho Shin, Lim Sun Suk, Kyungmi Kim, Lee Jihye, Sun Kim
| Challenge: | generative LLMs have been used by industries for various purposes, but limited resources and limited experience hinder their deployment and maintenance. |
| Approach: | They propose a taxonomy for sensitive search queries and outline approaches to generating generative LLMs. |
| Outcome: | The proposed model can be used to analyze sensitive queries from real users. |
Large Language Models are Temporal and Causal Reasoners for Video Question Answering (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown remarkable performances on a wide range of natural language understanding and generation tasks. |
| Approach: | They propose a framework that exploits linguistic shortcuts and mitigates 'linguistic bias' by flipping the source pair and target label to understand their complex relationships. |
| Outcome: | The proposed framework outperforms both LLMs-based and non-LLMs- based models on five challenging VideoQA benchmarks. |
ALPACA AGAINST VICUNA: Using LLMs to Uncover Memorization of LLMs (2025.naacl-long)
Copied to clipboard
Aly M. Kassem, Omar Mahmoud, Niloofar Mireshghallah, Hyunwoo Kim, Yulia Tsvetkov, Yejin Choi, Sherif Saad, Santu Rana
| Challenge: | Existing studies have shown that pre-trained LLMs emit training data up to 150 more often than in regular operation. |
| Approach: | They propose a black-box prompt optimization method where an attacker LLM agent uncovers higher levels of memorization in a victim agent . |
| Outcome: | The proposed method shows 23.7% more overlap with training data compared to state-of-the-art baselines. |
Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causes (2021.emnlp-main)
Copied to clipboard
| Challenge: | Empathy is a complex cognitive ability based on the reasoning of others’ affective states. |
| Approach: | They propose a method to infer emotion cause words from utterances without a word-level label and a novel method to make dialogue models focus on targeted words in the input during generation. |
| Outcome: | The proposed method improves multiple best-performing dialogue agents on generating more focused empathetic responses in terms of automatic and human evaluation. |
Generative Subgraph Retrieval for Knowledge Graph–Grounded Dialog Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for knowledge graph–grounded dialog generation fail to leverage the rich knowledge of pretrained language models. |
| Approach: | They propose a method for dialog generation that integrates dialog history with a knowledge graph. |
| Outcome: | The proposed method achieves state-of-the-art in knowledge graph–grounded dialog generation on OpenDialKG and KOMODIS datasets. |
Whose Voice, Whose Avatar? Gender Matching Bias in Multimodal AI Teammates (2026.findings-acl)
Copied to clipboard
| Challenge: | Multimodal Large Language Models are increasingly deployed as social agents . yet their ability to integrate conflicting identity cues remains underexplored . |
| Approach: | They audit gender bias in MLLMs that pair synthetic voices with avatars of varying gender presentation and visual fidelity. |
| Outcome: | The findings show that multimodal fairness is not monolithic . they show that models may appear unbiased on one dimension while enforcing stereotypes on another . |
How Robust are Fact Checking Systems on Colloquial Claims? (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing fact checking systems that perform well on colloquial claims significantly degenerate on collotic claims with the same semantics. |
| Approach: | They propose to transfer the styles of claims from FEVER into colloquialism to investigate fact checking systems on colloqual claims. |
| Outcome: | The proposed system significantly degenerates on colloquial claims with the same semantics. |
SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization (2023.emnlp-main)
Copied to clipboard
Hyunwoo Kim, Jack Hessel, Liwei Jiang, Peter West, Ximing Lu, Youngjae Yu, Pei Zhou, Ronan Bras, Malihe Alikhani, Gunhee Kim, Maarten Sap, Yejin Choi
| Challenge: | a dataset of 1.5 million conversations distilled from everyday spoken situations is limited in scale due to its associated costs. |
| Approach: | They propose to make SODA a publicly available, million-scale high-quality social dialogue dataset . they contextualize social commonsense knowledge from a knowledge graph to distill broad spectrum of social interactions . |
| Outcome: | The proposed dataset is the first publicly available, million-scale high-quality social dialogue dataset. |
XDetox: Text Detoxification with Token-Level Toxicity Explanations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for mitigating toxic content are black-box approaches, which results in limitations in modifying toxic tokens. |
| Approach: | They propose a method that integrates token-level toxicity explanations with the masking and infilling detoxification processes. |
| Outcome: | The proposed method outperforms baseline methods in fluency and toxicity reduction. |
ProsocialDialog: A Prosocial Backbone for Conversational Agents (2022.emnlp-main)
Copied to clipboard
Hyunwoo Kim, Youngjae Yu, Liwei Jiang, Ximing Lu, Daniel Khashabi, Gunhee Kim, Yejin Choi, Maarten Sap
| Challenge: | Existing dialogue systems fail to respond properly to potentially unsafe user utterances . existing systems either ignore or passively agree with unsafe content . |
| Approach: | They introduce a dataset to teach conversational agents to respond to problematic content following social norms. |
| Outcome: | The proposed dataset shows that ProsocialDialog generates more socially acceptable dialogues than existing models. |
Perceptions to Beliefs: Exploring Precursory Inferences for Theory of Mind in Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | While theory of mind (ToM) is naturally developed for humans in childhood, large language models (LLMs) exhibit inconsistency in ToM tasks, despite early reports of successful cases. |
| Approach: | They propose to evaluate human ToM precursors-perception inference and perception-to-belief inference-in large language models (LLMs) by annotating characters’ perceptions on ToMi and FANToM. |
| Outcome: | The proposed method significantly improves LLMs’ performance in false belief scenarios. |
Visual Interference in Speech Evaluation: Cultural Asymmetry and Cross-Modal Bias in MLLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | a new paradigm shifts the paradigm of speech processing from simple transcription to complex social reasoning. |
| Approach: | They construct a cross-modal dataset to examine cultural asymmetry in MLLMs . they find that ML models actively reproduce context-dependent sociolinguistic ideologies based on native audio . |
| Outcome: | The proposed model exhibits cultural asymmetry in anglophone and Korean contexts . the model reproduces sociolinguistic ideologies, consistent with Expectancy Violation Theory . |
CliniCAST: Benchmarking Acoustic Grounding and Text Dominance in Medical Triage (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent Large Audio-Language Models (LALMs) integrate acoustic capabilities into reasoning, yet whether they reliably ground clinical judgments in audible evidence remains unproven. |
| Approach: | They propose a benchmark that disentangles clinically meaningful acoustic cues from lexical content and speaker demographics. |
| Outcome: | Evaluating 5,856 synthetic samples across 12 disease conditions, the proposed model exhibits fragile acoustic grounding and pronounced "text dominance" failure mode. |