Papers by Yejin Kim
Subtle Risks, Critical Failures: A Framework for Diagnosing Physical Safety of LLMs for Embodied Decision Making (2025.emnlp-main)
Copied to clipboard
Yejin Son, Minseo Kim, Sungwoong Kim, Seungju Han, Jian Kim, Dongju Jang, Youngjae Yu, Chan Young Park
| Challenge: | Existing safety evaluations rely on coarse success rates and domain-specific setups, making it difficult to diagnose why and where these models fail. |
| Approach: | They propose a framework for systematically evaluating the physical safety of LLMs in embodied decision making. |
| Outcome: | The proposed framework assesses the physical safety of LLMs in embodied decision making. |
FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluations for theory of mind (ToM) use passive narratives that lack interactivity. |
| Approach: | They propose a benchmark to stress-test ToM within information-asymmetric conversational contexts via question answering. |
| Outcome: | The proposed benchmark is challenging for state-of-the-art language models, which perform significantly worse than humans even with chain-of thought reasoning or fine-tuning. |
Beyond Task-Oriented and Chitchat Dialogues: Proactive and Transition-Aware Conversational Agents (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current efforts to bridge the two modes of interaction are reactive, focusing on responding to user inputs rather than coordinating dialogue flows. |
| Approach: | They propose a dataset designed for transition-aware dialogue modeling that incorporates structurally diverse and integrated mode flows. |
| Outcome: | The proposed dataset outperforms baseline models in intent detection and mode transition handling. |
Don’t Judge a Language Model by Its Last Layer: Contrastive Learning with Layer-Wise Attention Pooling (2022.coling-1)
Copied to clipboard
| Challenge: | Recent pre-trained language models (PLMs) have shown competitive performance on many natural language processing tasks. |
| Approach: | They propose a pooling strategy which preserves layer-wise signals captured in each layer and learns digested linguistic features for downstream tasks. |
| Outcome: | The proposed method improves on standard semantic textual similarity and semantic search tasks. |
Socratic-MCTS: Test-Time Visual Reasoning by Asking the Right Questions (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent research in vision-language models has centered around the possibility of equipping them with implicit long-form chain-of-thought reasoning via distillation and reinforcement learning. |
| Approach: | They propose a Monte Carlo Tree Search-inspired algorithm that injects subquestion–subanswer pairs into the model’s output stream to elicit hidden knowledge and induce long reasoning traces. |
| Outcome: | The proposed method yields a 2% improvement on MMMU-PRO, including a significant 9% gain in Liberal Arts. |
PanicToCalm: A Proactive Counseling Agent for Panic Attacks (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for training such models are limited due to ethical and logistical issues. |
| Approach: | They propose a dataset that includes high-distress episodes constructed from first-person narratives and structured around the principles of Psychological First Aid. |
| Outcome: | The proposed model outperforms baseline models in counselor-side metrics and client affect improvement. |
Progressive Facial Granularity Aggregation with Bilateral Attribute-based Enhancement for Face-to-Speech Synthesis (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for generating speech from facial images rely on pre-trained visual encoders and fine-tune them to align with speech embeddings. |
| Approach: | They propose to derive corresponding voices from facial images using face-to-voice synthesis, which derives corresponding voice from facial image. |
| Outcome: | The proposed approach significantly improves face-voice congruence and synthesis stability. |
ALPACA AGAINST VICUNA: Using LLMs to Uncover Memorization of LLMs (2025.naacl-long)
Copied to clipboard
Aly M. Kassem, Omar Mahmoud, Niloofar Mireshghallah, Hyunwoo Kim, Yulia Tsvetkov, Yejin Choi, Sherif Saad, Santu Rana
| Challenge: | Existing studies have shown that pre-trained LLMs emit training data up to 150 more often than in regular operation. |
| Approach: | They propose a black-box prompt optimization method where an attacker LLM agent uncovers higher levels of memorization in a victim agent . |
| Outcome: | The proposed method shows 23.7% more overlap with training data compared to state-of-the-art baselines. |
Safeguarding RAG Pipelines with GMTP: A Gradient-based Masked Token Probability Method for Poisoned Document Detection (2025.findings-acl)
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) provides external knowledge for accurate and up-to-date responses, but external knowledge is vulnerable to poisoning and unauthorized injections. |
| Approach: | They propose a Gradient-based Masked Token Probability defense method to detect and filter out adversarially crafted documents by examining gradients of the retriever’s similarity function. |
| Outcome: | Experiments show that the proposed method eliminates over 90% of poisoned content while retaining relevant documents. |
Being Kind Isn’t Always Being Safe: Diagnosing Affective Hallucination in LLMs (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly engaged in emotionally vulnerable conversations that extend beyond information seeking to moments of personal distress. |
| Approach: | They propose AHaBench, a benchmark of 500 mental-health-related prompts with expert-informed reference responses, evaluated along three dimensions: Emotional Enmeshment, Illusion of Presence, and Fostering Overdependence. |
| Outcome: | The proposed model is based on 500 mental-health-related prompts with expert-informed reference responses and a 5K-instance preference dataset enabling direct preference optimization (DPO) for alignment with emotionally responsible behavior. |
Do MLLMs Capture How Interfaces Guide User Behavior? A Benchmark for Multimodal UI/UX Design Understanding (2026.acl-long)
Copied to clipboard
Jaehyun Jeon, Min Soo Kim, Janghan Yoon, Sumin Shim, Yejin Choi, Hanbin Kim, Dae Hyun Kim, Youngjae Yu
| Challenge: | Recent studies focus on surface-level features, overlooking how design choices influence user behavior at scale. |
| Approach: | They propose a benchmark for multimodal understanding of how UI/UX design affects user behavior built on 300 real-world UI image pairs from industry A/B tests. |
| Outcome: | The proposed benchmarks show that models exhibit limited understanding of the behavioral impact of UI/UX design. |
Improving Content Recommendation: Knowledge Graph-Based Semantic Contrastive Learning for Diversity and Cold-Start Users (2024.lrec-main)
Copied to clipboard
Yejin Kim, Scott Rome, Kevin Foley, Mayur Nankani, Rimon Melamed, Javier Morales, Abhay K. Yadav, Maria Peifer, Sardar Hamidian, H. Howie Huang
| Challenge: | Current approaches focus on improving ranking performance at the cost of escalating complexity and complicating the task. |
| Approach: | They propose a hybrid multi-task learning approach that trains on user-item and item-i item interactions. |
| Outcome: | The proposed approach improves accuracy, relevance, and diversity of user recommendations even for cold-start users. |
Prompts have evil twins (2024.emnlp-main)
Copied to clipboard
| Challenge: | We find that many natural-language prompts can be replaced by corresponding unintelligible prompts that provably elicit similar behavior in language models. |
| Approach: | They find that natural-language prompts can be replaced by corresponding unintelligible prompts that elicit similar behavior in language models. |
| Outcome: | The proposed prompts are obfuscated and uninterpretable but mimic the original natural-language prompts . the problem has applications of independent interest, the authors argue . |
MALLM-GAN: Multi-Agent Large Language Model as Generative Adversarial Network for Synthesizing Tabular Data (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing models for tabular data generation require large amounts of data to train effectively. |
| Approach: | They propose a framework to generate tabular data powered by large language models that emulates a Generative Adversarial Network. |
| Outcome: | The proposed framework outperforms state-of-the-art models while keeping privacy of real data. |
Multimodal UNcommonsense: From Odd to Ordinary and Ordinary to Odd (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Multimodal UNcommonsense (MUN) is a benchmark designed to evaluate models’ ability to handle scenarios that deviate from typical visual or contextual expectations. |
| Approach: | They propose a retrieval-based in-context learning framework that transfers reasoning capabilities from larger models to smaller ones without additional training. |
| Outcome: | The proposed method improves on baseline ICL methods by 8.3% over previous methods. |
Leveraging the Interplay between Syntactic and Acoustic Cues for Optimizing Korean TTS Pause Formation (2024.lrec-main)
Copied to clipboard
| Challenge: | despite recent advances in speech synthesis, the focus of research has been on high-resource languages like English. |
| Approach: | They propose a framework that incorporates modeling of syntactic and acoustic cues associated with pausing patterns. |
| Outcome: | The proposed framework generates natural speech even for longer and intricate out-of-domain sentences, despite training on short audio clips. |
Investigating Counterfactual Unfairness in LLMs towards Identities through Humor (2026.acl-long)
Copied to clipboard
Shubin Kim, Yejin Son, Junyeong Park, Keummin Ka, Seungbeen Lee, Jaeyoung Lee, Hyeju Jang, Alice Oh, Youngjae Yu
| Challenge: | Large Language Models (LLMs) absorb social and cultural biases embedded in vast web-scale corpora and are increasingly deployed in high-stakes domains such as hiring, education, and law. |
| Approach: | They propose a framework to investigate counterfactual unfairness through humor by observing how the model’s responses change when we swap who speaks and who is addressed while holding other factors constant. |
| Outcome: | The proposed framework covers humor generation refusal, speaker intention inference, and relational/societal impact prediction tasks. |
SQuARe: A Large-Scale Dataset of Sensitive Questions and Acceptable Responses Created through Human-Machine Collaboration (2023.acl-long)
Copied to clipboard
Hwaran Lee, Seokhee Hong, Joonsuk Park, Takyoung Kim, Meeyoung Cha, Yejin Choi, Byoungpil Kim, Gunhee Kim, Eun-Ju Lee, Yong Lim, Alice Oh, Sangchul Park, Jung-Woo Ha
| Challenge: | Existing studies focus on coping with social harms that large language models pose . however, discussions on sensitive issues can become toxic even if the users are well-intentioned. |
| Approach: | They propose to use Korean dataset to test whether LLMs can generate offensive content and propagate prejudices. |
| Outcome: | The proposed dataset shows that acceptable response generation improves for HyperCLOVA and GPT-3. |
SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization (2023.emnlp-main)
Copied to clipboard
Hyunwoo Kim, Jack Hessel, Liwei Jiang, Peter West, Ximing Lu, Youngjae Yu, Pei Zhou, Ronan Bras, Malihe Alikhani, Gunhee Kim, Maarten Sap, Yejin Choi
| Challenge: | a dataset of 1.5 million conversations distilled from everyday spoken situations is limited in scale due to its associated costs. |
| Approach: | They propose to make SODA a publicly available, million-scale high-quality social dialogue dataset . they contextualize social commonsense knowledge from a knowledge graph to distill broad spectrum of social interactions . |
| Outcome: | The proposed dataset is the first publicly available, million-scale high-quality social dialogue dataset. |
ProsocialDialog: A Prosocial Backbone for Conversational Agents (2022.emnlp-main)
Copied to clipboard
Hyunwoo Kim, Youngjae Yu, Liwei Jiang, Ximing Lu, Daniel Khashabi, Gunhee Kim, Yejin Choi, Maarten Sap
| Challenge: | Existing dialogue systems fail to respond properly to potentially unsafe user utterances . existing systems either ignore or passively agree with unsafe content . |
| Approach: | They introduce a dataset to teach conversational agents to respond to problematic content following social norms. |
| Outcome: | The proposed dataset shows that ProsocialDialog generates more socially acceptable dialogues than existing models. |
Reading Books is Great, But Not if You Are Driving! Visually Grounded Reasoning about Defeasible Commonsense Norms (2023.emnlp-main)
Copied to clipboard
Seungju Han, Junhyeok Kim, Jack Hessel, Liwei Jiang, Jiwan Chung, Yejin Son, Yejin Choi, Youngjae Yu
| Challenge: | NormLens is a visual-grounded framework for understanding commonsense norms . state-of-the-art models are not well-aligned with human annotation, we show . |
| Approach: | They propose a visual-grounded framework to study commonsense norms by NormLens . they find that models are not well-aligned with human annotation . |
| Outcome: | The proposed model judgments and explanations are not well-aligned with human annotations. |
BlendX: Complex Multi-Intent Detection with Blended Patterns (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing datasets such as MixATIS and MixSNIPS have limitations in their formulation. |
| Approach: | They propose a set of multi-intent detection datasets that feature more diverse patterns than their predecessors. |
| Outcome: | The proposed datasets feature more diverse patterns than their predecessors and are more complex and diverse than existing datasets. |
Audio-Based Linguistic Feature Extraction for Enhancing Multi-lingual and Low-Resource Text-to-Speech (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to synthesize speech for low-resource languages require a substantial amount of source language corpora to generate the linguistic knowledge that can be reused for speech synthesis. |
| Approach: | They propose a method that extracts linguistic features from audio input while effectively filtering out miscellaneous acoustic information including speaker-specific attributes like timbre. |
| Outcome: | The proposed method extracts linguistic features from audio input while effectively filtering out miscellaneous acoustic information including speaker-specific attributes like timbre. |
The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models (2025.naacl-long)
Copied to clipboard
Seungone Kim, Juyoung Suk, Ji Yong Cho, Shayne Longpre, Chaeeun Kim, Dongkeun Yoon, Guijin Son, Yejin Cho, Sheikh Shafayat, Jinheon Baek, Sue Hyun Park, Hyeonbin Hwang, Jinkyung Jo, Hyowon Cho, Haebin Shin, Seongyun Lee, Hanseok Oh, Noah Lee, Namgyu Ho, Se June Joo, Miyoung Ko, Yoonjoo Lee, Hyungjoo Chae, Jamin Shin, Joel Jang, Seonghyeon Ye, Bill Yuchen Lin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, Minjoon Seo
| Challenge: | a recent study evaluated language models using abstract evaluation criteria that lack the flexibility and granularity of human assessment. |
| Approach: | They propose a benchmark to evaluate nine distinct language models' capabilities . they use instance-specific evaluation criteria to mirror human evaluation . |
| Outcome: | The proposed benchmark evaluates nine distinct capabilities of language models across 77 tasks. |
Perceptions to Beliefs: Exploring Precursory Inferences for Theory of Mind in Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | While theory of mind (ToM) is naturally developed for humans in childhood, large language models (LLMs) exhibit inconsistency in ToM tasks, despite early reports of successful cases. |
| Approach: | They propose to evaluate human ToM precursors-perception inference and perception-to-belief inference-in large language models (LLMs) by annotating characters’ perceptions on ToMi and FANToM. |
| Outcome: | The proposed method significantly improves LLMs’ performance in false belief scenarios. |
MIRROR: Multimodal Cognitive Reframing Therapy for Rolling with Resistance (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have explored the use of large language models (LLMs) in psychotherapy, however text-based cognitive behavioral therapy models struggle with client resistance, which weakens therapeutic alliance. |
| Approach: | They propose a multimodal approach that incorporates nonverbal cues and a synthetic dataset that pairs each client’s statements with corresponding facial images to train vision language models. |
| Outcome: | The proposed approach outperforms existing text-based cognitive behavioral therapy models in managing client resistance and fostering therapeutic alliance. |
Prompt-Guided Selective Masking Loss for Context-Aware Emotive Text-to-Speech (2025.findings-naacl)
Copied to clipboard
| Challenge: | Emotional dialogue speech synthesis (EDSS) aims to generate expressive speech by leveraging the dialogue context between interlocutors. |
| Approach: | They propose a large language model to generate holistic emotion tags based on prior dialogue context and pinpoint key words in the target utterance that align with the predicted emotional state. |
| Outcome: | The proposed method improves emotional expressiveness and facilitates automatic emotion speech generation during inference. |
Zero-shot Multimodal Document Retrieval via Cross-modal Question Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing multimodal large language models struggle when faced with unseen domains or languages. |
| Approach: | They propose a framework that leverages the broad knowledge of an MLLM to generate cross-modal pre-questions (preQs) before retrieval. |
| Outcome: | Experiments show that PREMIR outperforms existing retrievers on out-of-distribution benchmarks, including closed-domain and multilingual settings, outperforming strong baselines across all metrics. |