Papers by Gunhee Kim
A Hierarchical Latent Structure for Variational Conversation Modeling (N18-1)
Copied to clipboard
| Challenge: | Variational autoencoders suffer from the notorious degeneration problem, according to a new study . utterance drop regularization is an important feature of the hierarchical RNNs . |
| Approach: | They propose a variational hierarchical conversation RNN framework that exploits latent variables and an utterance drop regularization to exploit latent variable. |
| Outcome: | The proposed model outperforms state-of-the-art models on Cornell Movie Dialog and Ubuntu Dialog Corpus. |
Who Wrote this Code? Watermarking for Code Generation (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods to detect machine-generated text by embedding watermarks fail to function appropriately in code generation tasks due to the task’s nature of having low entropy. |
| Approach: | They propose a logit-modifying watermark method which enhances detection ability and mitigates code quality degeneration by removing low-entropy segments at generating and detecting watermarks. |
| Outcome: | The proposed method outperforms baseline methods in detecting machine-generated code text while preserving code quality. |
FlashAdventure: A Benchmark for GUI Agents Solving Full Story Arcs in Diverse Adventure Games (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing game benchmarks lack diversity and evaluate GUI agents on completing entire storylines. |
| Approach: | They propose a benchmark of 34 Flash-based adventure games to test full story arc completion and tackle observation-behavior gap. |
| Outcome: | The proposed benchmarks show GUI agents struggle with full story arc completion while others improve on observation-behavior gaps. |
When Should Dense Retrievers Be Updated in Evolving Corpora? Detecting Out-of-Distribution Corpora Using GradNormIR (2025.findings-acl)
Copied to clipboard
Dayoon Ko, Jinyoung Kim, Sohyeon Kim, Jinhyuk Kim, Jaehoon Lee, Seonghak Song, Minyoung Lee, Gunhee Kim
| Challenge: | Dense retrievers encode text into embeddings to retrieve relevant documents . however, real-world corpora evolve, resulting in degraded retrieval performance . identifying when a dense retriever requires an update is critical for robust retrieval systems . |
| Approach: | They propose a task of predicting whether a corpus is out-of-distribution (OOD) relative to a dense retriever before indexing. |
| Outcome: | The proposed method detects whether a corpus is out-of-distribution (OOD) relative to a dense retriever before indexing. |
FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluations for theory of mind (ToM) use passive narratives that lack interactivity. |
| Approach: | They propose a benchmark to stress-test ToM within information-asymmetric conversational contexts via question answering. |
| Outcome: | The proposed benchmark is challenging for state-of-the-art language models, which perform significantly worse than humans even with chain-of thought reasoning or fine-tuning. |
WoW-Bench: Evaluating Fine-Grained Acoustic Perception in Audio-Language Models via Marine Mammal Vocalizations (2026.findings-acl)
Copied to clipboard
| Challenge: | Large audio-language models extend language understanding into the auditory domain, yet their ability to perform low-level listening, such as pitch and duration detection, remains underexplored. |
| Approach: | They propose a global benchmark to evaluate low-level auditory perception and cognition using marine mammal vocalizations to better assess models’ low- level listening. |
| Outcome: | The proposed models show performance far below human levels, indicating a need for stronger auditory grounding in LALMs. |
Will I Sound Like Me? Improving Persona Consistency in Dialogues through Pragmatic Self-Consciousness (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for improving consistency often train with additional NLI labels or attach trained extra modules to the generative agent. |
| Approach: | They propose to encode personas into dialogue embeddings and a persona-conditioned dialogue dataset to improve persona consistency. |
| Outcome: | The proposed approach can enforce dialogue agents to refrain from contradictions and improve consistency of existing models. |
Abstractive Summarization of Reddit Posts with Multi-level Memory Networks (N19-1)
Copied to clipboard
| Challenge: | Abstractive summarization methods suffer from inferior performance compared to extractive methods. |
| Approach: | They propose a reddit TIFU dataset and a new abstractive summarization model . they use multi-level memory networks to store information from different levels of abstraction . |
| Outcome: | The proposed model outperforms state-of-the-art summarization models with multi-level memory . the proposed dataset is highly abstractive and outperformed existing models with the proposed model . |
Is a Peeled Apple Still Red? Evaluating LLMs’ Ability for Conceptual Combination with Property Type (2025.naacl-long)
Copied to clipboard
| Challenge: | Conceptual combination is a cognitive process that merges basic concepts, enabling the creation of complex expressions. |
| Approach: | They propose to use a Conceptual Combination with Property Type dataset to evaluate LLMs for conceptual combination thoroughly. |
| Outcome: | The proposed method improves performance in all generative tasks. |
Text2Chart31: Instruction Tuning for Chart Generation with Automatic Feedback (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets do not cover full range of chart types, such as 3D, volumetric, and gridded charts. |
| Approach: | They propose a hierarchical pipeline and a new dataset for chart generation that leverages the relationships within rich datasets. |
| Outcome: | The proposed method outperforms open-source models and is comparable to state-of-the-art proprietary models in data visualization tasks. |
LPOI: Listwise Preference Optimization for Vision Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for aligning large VLMs with human preferences often overfit to textual information or exacerbate hallucinations. |
| Approach: | They propose an object-aware listwise preference optimization for reducing hallucinations in VLMs . they mask a critical object in an image and interpolate the masked region to form more complete images . |
| Outcome: | The proposed method outperforms existing methods in reducing hallucinations and enhancing performance on MMHalBench, AMBER, and Object HalBench. |
DynamicER: Resolving Emerging Mentions to Dynamic Entities for RAG (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing entity linking models struggle to link new expressions to entities in the dynamic nature of human language. |
| Approach: | They propose a task to resolve emerging mentions to dynamic entities and a benchmark to evaluate their model's adaptability to new expressions. |
| Outcome: | The proposed method outperforms baselines on QA task with resolved mentions and improves retrieval-augmented generation performance. |
Can LLMs Deceive CLIP? Benchmarking Adversarial Compositionality of Pre-trained Multimodal Representation via Text Updates (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in multimodal systems have demonstrated remarkable capabilities in generating multimodal content from multimodal inputs. |
| Approach: | They propose a benchmark that leverages large language models to generate deceptive text samples to exploit compositional vulnerabilities across different modalities. |
| Outcome: | The proposed approach exploits compositional vulnerabilities across images, videos, and audios. |
MPCHAT: Towards Multimodal Persona-Grounded Conversation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing research on persona-based dialogue has focused on textual persona that delivers personal facts or personalities, but image modality can reveal the speaker’s personal characteristics and experiences in episodic memory. |
| Approach: | They propose a multimodal persona-based dialogue dataset which extends persona with both text and images to contain episodic memories. |
| Outcome: | The proposed dataset extends persona with text and images to contain episodic memories. |
Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causes (2021.emnlp-main)
Copied to clipboard
| Challenge: | Empathy is a complex cognitive ability based on the reasoning of others’ affective states. |
| Approach: | They propose a method to infer emotion cause words from utterances without a word-level label and a novel method to make dialogue models focus on targeted words in the input during generation. |
| Outcome: | The proposed method improves multiple best-performing dialogue agents on generating more focused empathetic responses in terms of automatic and human evaluation. |
KoSBI: A Dataset for Mitigating Social Bias Risks Towards Safer Large Language Model Applications (2023.acl-industry)
Copied to clipboard
| Challenge: | Existing research and resources are not readily applicable in South Korea due to the differences in language and culture, both of which significantly affect the biases and targeted demographic groups. |
| Approach: | They propose a social bias dataset of 34k pairs of contexts and sentences in Korean covering 72 demographic groups in 15 categories. |
| Outcome: | The proposed dataset reduces social biases by 16.47%p on average for HyperClova (30B and 82B), and GPT-3. |
TimeChara: Evaluating Point-in-Time Character Hallucination of Role-Playing Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can be used to simulate human behaviors, but point-in-time role-playing is a key component of fandom role-players. |
| Approach: | They propose a benchmark to evaluate point-in-time character hallucination in role-playing LLMs. |
| Outcome: | The proposed method reduces point-in-time character hallucinations effectively by decomposing reasoning steps and using narrative experts. |
Recursion of Thought: A Divide-and-Conquer Approach to Multi-Context Reasoning with Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to generate intermediate steps (CoT) are limited by the maximum context size due to various reasons. |
| Approach: | They propose a new inference framework that introduces several special tokens that the models can output to trigger context-related operations. |
| Outcome: | Extensive experiments with multiple architectures including GPT-3 show that the proposed framework significantly improves LMs’ inference capability. |
GrowOVER: How Can LLMs Adapt to Growing Real-World Knowledge? (2024.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge-based datasets are outdated due to the rapid evolution of knowledge. |
| Approach: | They propose a retrieval-interactive language model framework that evaluates and reflects on its answers for further re-retrieval. |
| Outcome: | The proposed framework performs comparably to or surpasses continuously trained language models. |
How Robust are Fact Checking Systems on Colloquial Claims? (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing fact checking systems that perform well on colloquial claims significantly degenerate on collotic claims with the same semantics. |
| Approach: | They propose to transfer the styles of claims from FEVER into colloquialism to investigate fact checking systems on colloqual claims. |
| Outcome: | The proposed system significantly degenerates on colloquial claims with the same semantics. |
AudioCaps: Generating Captions for Audios in The Wild (N19-1)
Copied to clipboard
| Challenge: | a dataset of 46K audio clips with human-written text pairs is used to generate captions for audio . the task of translating a multimedia input source into natural language has been extensively studied over the past few years . |
| Approach: | They propose a top-down multi-scale encoder and aligned semantic attention for audio captioning. |
| Outcome: | The proposed captions are faithful to audio inputs and better than existing models. |
Think, Verbalize, then Speak: Bridging Complex Thoughts and Comprehensible Speech (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to decouple LLMs from spoken communication produce suboptimal results due to mismatches between optimal textual and verbal delivery. |
| Approach: | They propose a framework that decouples reasoning from spoken delivery to preserve the full reasoning capacity of LLMs. |
| Outcome: | The proposed framework preserves full reasoning capacity of large language models . it improves speech naturalness and conciseness with minimal impact on reasoning . |
mRedditSum: A Multimodal Abstractive Summarization Dataset of Reddit Threads with Images (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing summarization datasets do not cover multimodal discussions, multiple modalities, or both . mRedditSum consists of 3,033 discussion threads and images with human-written summaries. |
| Approach: | They propose a multimodal discussion summarization dataset that annotates 3,033 discussion threads with a human-written summary. |
| Outcome: | The proposed method outperforms existing models and serves as competitive baseline for future work. |
See It All: Contextualized Late Aggregation for 3D Dense Captioning (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent approaches to 3D dense captioning struggle with contradicting objectives . SIA generates captions with different region of interest and aggregates them afterwards . |
| Approach: | They propose a transformer pipeline that engages in 3D dense captioning with a new paradigm . SIA decodes two sets of queries—context query and instance query—and then aggregates them afterwards . |
| Outcome: | The proposed pipeline generates captions with different region of interest and aggregates them afterwards to enhance local-global sensitivity. |
SQuARe: A Large-Scale Dataset of Sensitive Questions and Acceptable Responses Created through Human-Machine Collaboration (2023.acl-long)
Copied to clipboard
Hwaran Lee, Seokhee Hong, Joonsuk Park, Takyoung Kim, Meeyoung Cha, Yejin Choi, Byoungpil Kim, Gunhee Kim, Eun-Ju Lee, Yong Lim, Alice Oh, Sangchul Park, Jung-Woo Ha
| Challenge: | Existing studies focus on coping with social harms that large language models pose . however, discussions on sensitive issues can become toxic even if the users are well-intentioned. |
| Approach: | They propose to use Korean dataset to test whether LLMs can generate offensive content and propagate prejudices. |
| Outcome: | The proposed dataset shows that acceptable response generation improves for HyperCLOVA and GPT-3. |
SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization (2023.emnlp-main)
Copied to clipboard
Hyunwoo Kim, Jack Hessel, Liwei Jiang, Peter West, Ximing Lu, Youngjae Yu, Pei Zhou, Ronan Bras, Malihe Alikhani, Gunhee Kim, Maarten Sap, Yejin Choi
| Challenge: | a dataset of 1.5 million conversations distilled from everyday spoken situations is limited in scale due to its associated costs. |
| Approach: | They propose to make SODA a publicly available, million-scale high-quality social dialogue dataset . they contextualize social commonsense knowledge from a knowledge graph to distill broad spectrum of social interactions . |
| Outcome: | The proposed dataset is the first publicly available, million-scale high-quality social dialogue dataset. |
ProsocialDialog: A Prosocial Backbone for Conversational Agents (2022.emnlp-main)
Copied to clipboard
Hyunwoo Kim, Youngjae Yu, Liwei Jiang, Ximing Lu, Daniel Khashabi, Gunhee Kim, Yejin Choi, Maarten Sap
| Challenge: | Existing dialogue systems fail to respond properly to potentially unsafe user utterances . existing systems either ignore or passively agree with unsafe content . |
| Approach: | They introduce a dataset to teach conversational agents to respond to problematic content following social norms. |
| Outcome: | The proposed dataset shows that ProsocialDialog generates more socially acceptable dialogues than existing models. |
Behavior-SD: Behaviorally Aware Spoken Dialogue Generation with Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Spoken dialogues lack explicit modeling of behavior traits that are often overlooked in language models . et al.: our work opens new possibilities for developing behaviorally-aware dialogue systems . |
| Approach: | They propose a large-scale dataset with over 100K spoken dialogues (2,164 hours) they propose BeDLM, the first dialogue model capable of generating natural conversations . |
| Outcome: | The proposed model outperforms baseline models in generating natural dialogues . the proposed model can generate natural conversations conditioned on behavioral and narrative contexts - a key feature of spoken language models . |
Can Language Models Laugh at YouTube Short-form Videos? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets that focus on verbal cues and focus on short-form funny videos focus on focusing on verbs and visual cue. |
| Approach: | They curate a user-generated dataset of 10K multimodal funny videos from YouTube and annotate each video with timestamps and explanations for funny moments. |
| Outcome: | The proposed dataset improves the ability of large language models to understand humor. |