Papers by Hwanhee Lee
MARCH: Evaluating the Intersection of Ambiguity Interpretation and Multi-hop Inference (2026.findings-acl)
Copied to clipboard
Jeonghyun Park, Ingeol Baek, Seunghyun Yoon, Haeun Jang, Aparna Garimella, Akriti Jain, Nedim Lipka, Hwanhee Lee
| Challenge: | Existing benchmarks on multi-hop QA focus on single-hop and layered ambiguity, but they focus on ambiguous questions . ambiguities can arise at any stage, complicating the reasoning process . |
| Approach: | They propose a benchmark to evaluate ambiguity in multi-hop question answering . they propose MARCH, which uses 2,209 carefully annotated questions . |
| Outcome: | The proposed framework outperforms existing approaches and significantly outperfies existing frameworks. |
How Do Large Vision-Language Models See Text in Image? Unveiling the Distinctive Role of OCR Heads (2025.emnlp-main)
Copied to clipboard
| Challenge: | Despite advances in Large Vision Language Models, a gap remains in their interpretability and performance. |
| Approach: | They identify the Optical Character Recognition Head (OCR Head) heads that are more efficient at recognizing text from images. |
| Outcome: | The Optical Character Recognition Head (OCR Head) is identified as the most efficient head for recognizing text from images. |
Dynamic Order Template Prediction for Generative Aspect-Based Sentiment Analysis (2025.acl-short)
Copied to clipboard
| Challenge: | Aspect-based sentiment analysis (ABSA) assesses sentiments towards aspects within texts, resulting in detailed sentiment tuples. |
| Approach: | They propose a dynamic order template method that dynamically creates an order template that contains only the necessary views for each instance. |
| Outcome: | The proposed method improves F1 scores on ASQP and ACOS datasets while significantly reducing inference time. |
AdvisorQA: Towards Helpful and Harmless Advice-seeking Question Answering with Collective Intelligence (2025.naacl-long)
Copied to clipboard
| Challenge: | AdvisorQA aims to improve LLMs’ capability to offer advice for deeply subjective concerns, utilizing the LifeProTips Reddit forum. |
| Approach: | They propose a dataset to train LLMs' ability to offer advice for deeply subjective concerns, utilizing the LifeProTips Reddit forum. |
| Outcome: | The proposed model improves usefulness through automatic metric, GPT-4 and human evaluations, and expands independent evaluation axis to include harmlessness. |
UMIC: An Unreferenced Metric for Image Captioning via Contrastive Learning (2021.acl-short)
Copied to clipboard
| Challenge: | BERTScore and other text generation metrics do not use reference captions to evaluate image captions. |
| Approach: | They propose a new metric which does not require reference captions to evaluate image captions . they train UMIC to discriminate negative captions via contrastive learning . |
| Outcome: | The proposed metric has higher correlation than previous metrics that require multiple references. |
KPQA: A Metric for Generative Question Answering Using Keyphrase Weights (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing n-gram similarity metrics fail to discriminate the incorrect answers due to the free-form of the answer. |
| Approach: | They propose a new metric that assigns different weights to each token via keyphrase prediction to judge the correctness of GenQA. |
| Outcome: | The proposed metric has a significantly higher correlation with human judgments than existing metrics in various datasets. |
Probing-RAG: Self-Probing to Guide Language Models in Selective Document Retrieval (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to optimize retrieve-and-generate processes for real-world scenarios may not be optimal for large language models. |
| Approach: | They propose a Probing-RAG which utilizes hidden state representations from the intermediate layers of language models to adaptively determine the necessity of additional retrievals for a given query. |
| Outcome: | The proposed method outperforms previous methods while reducing the number of redundant retrieval steps. |
Investigating Language Preference of Multilingual RAG Systems (2025.findings-acl)
Copied to clipboard
| Challenge: | Empirical results show that DKM-RAG mitigates language preference in generation and enhances performance across diverse linguistic settings. |
| Approach: | They propose a framework that integrates translated passages with internal knowledge to overcome these issues. |
| Outcome: | The proposed framework mitigates language preference in generation and enhances performance across diverse linguistic settings. |
Keep Security! Benchmarking Security Policy Preservation in Large Language Model Contexts Against Indirect Attacks in Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly deployed in sensitive domains . large-scale benchmarks for contextual security preservation against attacks remain lacking . |
| Approach: | They evaluate 10 Large Language Models on a benchmark dataset to assess their adherence to contextual non-disclosure policies. |
| Outcome: | The proposed model fails to adhere to user-defined security policies in question answering . the model fails in indirect attacks, especially when it violates user-definable policies . |
Critic-Guided Decoding for Controlled Text Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has demonstrated reinforcement learning and weighted decoding as effective approaches to achieve a higher level of language control and quality with pros and cons. |
| Approach: | They propose a method that combines reinforcement learning and weighted decoding to train a critic from reward models. |
| Outcome: | The proposed method generates more coherent and well-controlled texts than previous methods on three controlled generation tasks, topic control, sentiment control, and detoxification. |
Enhancing Multilingual RAG Systems with Debiased Language Preference-Guided Query Fusion (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that mRAGs exhibit a perceived preference for high-resource languages, particularly English. |
| Approach: | They propose a debiased language preference metric to explicitly factor out structural priors . they propose mRAG framework that leverages monolingual alignment to optimize cross-lingual retrieval and generation. |
| Outcome: | The proposed framework outperforms baselines for English pivoting and mRAG in multiple languages. |
Dialogizer: Context-aware Conversational-QA Dataset Generation from Textual Sources (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing dialog inpainting methods generate ConvQA datasets with low contextual relevance due to insufficient learning of question-answer alignment. |
| Approach: | They propose a dialog inpainting method that generates ConvQA datasets from documents . they propose re-ranking tasks and a framework that generate contextually relevant questions . |
| Outcome: | The proposed framework generates ConvQA datasets with high contextual relevance from textual sources. |
LifeTox: Unveiling Implicit Toxicity in Life Advice (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing safety benchmarks and red teaming prompts fail to capture implicit toxicity in complex real-life advice-seeking scenarios. |
| Approach: | They propose a dataset designed for identifying implicit toxicity within advice-seeking scenarios. |
| Outcome: | The proposed dataset matches or surpasses the zero-shot performance of large language models in toxicity classification tasks. |
ToDi: Token-wise Distillation via Fine-Grained Divergence Control (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) offer impressive performance but are impractical for resource-constrained deployment due to high latency and energy consumption. |
| Approach: | They propose a method that adaptively combines FKL and RKL per token using a sigmoid-based weighting function derived from the teacher-student probability log-ratio. |
| Outcome: | The proposed method outperforms baselines using uniform or less granular strategies across instruction-following benchmarks. |
SAFE-SQL: Self-Augmented In-Context Learning with Fine-grained Example Selection for Text-to-SQL (2025.emnlp-main)
Copied to clipboard
| Challenge: | Text-to-SQL aims to convert natural language questions into executable SQL queries. |
| Approach: | They propose a framework that generates and filters self-augmented examples for SQL generation . using self-generated examples, they surpass previous zero-shot and few-shot frameworks . |
| Outcome: | The proposed framework surpasses the previous zero-shot and few-shot frameworks, achieving higher execution accuracy. |
KoCoSa: Korean Context-aware Sarcasm Detection Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | Sarcasm is a form of verbal irony where someone says the opposite of what they mean . misunderstanding this sarcasm may lead to fatal errors in dialogue systems . |
| Approach: | They propose a dataset for the Korean dialogue sarcasm detection task that uses 12.8K daily Korean dialogues and the labels on the last response. |
| Outcome: | The proposed system outperforms strong baselines like large language models in the Korean sarcasm detection task. |
Task-specific Compression for Multi-task Language Models using Attribution-based Pruning (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing compression methods for multi-task language models use large number of parameter parameters even when performing only a specific task. |
| Approach: | They propose a training-free compression method for multi-task language models using pruning method . they use an attribution method to determine which neurons are essential for performing a specific task . |
| Outcome: | The proposed method outperforms baseline pruning methods on six widely-used datasets. |
Exploring Persona Sentiment Sensitivity in Personalized Dialogue Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Personalized dialogue systems have advanced with the integration of user-specific personas into large language models (LLMs). |
| Approach: | They propose a dialogue generation approach that explicitly accounts for persona polarity by combining a turn-based generation strategy with a profile ordering mechanism and sentiment-aware prompting. |
| Outcome: | The proposed approach accounts for persona polarity by combining a turn-based generation strategy with a profile ordering mechanism and sentiment-aware prompting. |
Asking Clarification Questions to Handle Ambiguity in Open-Domain QA (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Ambiguous questions persist in open-domain question answering because formulating a precise question with a unique answer is often challenging. |
| Approach: | They propose to ask a clarification question where the user’s response will help identify the interpretation that best aligns with the user's intention. |
| Outcome: | The proposed approach achieves F1 of 61.3, 25.1, and 40.5 on the three tasks, demonstrating the need for further improvements while providing competitive baselines for future work. |
Which Retain Set Matters for LLM Unlearning? A Case Study on Entity Unlearning (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are prone to retaining unauthorized or sensitive information from their training data, which raises privacy concerns. |
| Approach: | They propose to use a group of queries that share similar syntactic structures with the data targeted for removal to investigate the effects of unlearning on various subsets of the retain set. |
| Outcome: | The proposed method reduces the retention set, the portion of training data that is not targeted for removal, and improves model performance across subsets. |
IterCQR: Iterative Conversational Query Reformulation with Retrieval Guidance (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for conversational query reformulation depend on human annotations. |
| Approach: | They propose a method that reformulates context-dependent conversational queries without relying on human rewrites. |
| Outcome: | The proposed method shows state-of-the-art performance on two widely-used datasets. |
FIZZ: Factual Inconsistency Detection by Zoom-in Summary and Zoom-out Document (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for evaluating factual consistency in abstractive summarization systems have significant limitations, especially on refinement and interpretability. |
| Approach: | They propose a method for detecting summary factual inconsistency based on fine-grained atomic facts decomposition and adaptive granularity expansion. |
| Outcome: | The proposed method outperforms existing systems on the AGGREFACT benchmark dataset and achieves state-of-the-art performance. |
Dynamic Label Name Refinement for Few-Shot Dialogue Intent Classification (2025.acl-short)
Copied to clipboard
| Challenge: | Current intent classification systems face significant challenges due to the vast number of possible intents and significant semantic overlap among similar intent classes. |
| Approach: | They propose a dynamic label refinement method that retrieves relevant examples for a test input and leverages a large language model to dynamically refine intent labels based on semantic understanding. |
| Outcome: | The proposed method resolves confusion between semantically similar intents and generates more interpretable intent labels. |
Masked Summarization to Generate Factually Inconsistent Summaries for Improved Factual Consistency Checking (2022.findings-naacl)
Copied to clipboard
| Challenge: | Abstractive summarization systems generate paraphrases, but they often contain information inconsistent with the source text. |
| Approach: | They propose to generate factually inconsistent summaries using source texts and reference summary with key information masked to train a factual consistency classifier. |
| Outcome: | The proposed method outperforms existing models and shows a competitive correlation with human judgments. |
Personality Editing for Language Models through Adjusting Self-Referential Queries (2026.eacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are integral to conversational agents and content creation, but they lack robustness and require large-scale training data to achieve significant improvements in personality alignment. |
| Approach: | They propose a method that introduces adjustment queries where self-referential statements grounded in psychological constructs are treated analogously to factual knowledge to enable direct editing of personality-related responses. |
| Outcome: | The proposed method improves personality alignment across personality dimensions and requires only 12 editing samples to achieve significant improvements. |
Kosmic: Korean Text Similarity Metric Reflecting Honorific Distinctions (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for text similarity measurement focus on the semantic dimension, neglecting the unique linguistic attributes found in languages like Korean. |
| Approach: | They propose a Korean text-similarity metric that encompasses the semantic and tonal facets of a given text pair. |
| Outcome: | The proposed method outperforms existing methods in Korean and other languages . it identifies which methods preserve semantics and tone while preserving similarity . |
Doc-PP: Document Policy Preservation Benchmark for Large Vision-Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing safety research focuses on implicit social norms or text-only settings, overlooking the complexities of multimodal documents. |
| Approach: | They propose a benchmark to assess the safety of large vision-Language Models (LVLMs) they propose 'Document Policy Preservation Benchmark' to assess document policy compliance. |
| Outcome: | The proposed framework outperforms standard prompting defenses in the evaluation of multimodal documents. |
QACE: Asking Questions to Evaluate an Image Caption (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing metric for image captioning evaluation is based on n-gram similarity metrics but these fail to capture semantic errors in captions. |
| Approach: | They propose a new metric based on Question Answering for Caption Evaluation to evaluate image captioning based upon Question Generation and Question Answers systems. |
| Outcome: | The proposed metric is multi-modal, reference-less and explainable. |