Papers by Dongryeol Lee
Return of EM: Entity-driven Answer Set Expansion for QA Evaluation (2025.coling-main)
Copied to clipboard
| Challenge: | Recent studies show that using large language models (LLMs) is the most reliable method to evaluate QA models, but suffers from limited interpretability, high cost, and environmental harm. |
| Approach: | They propose to use soft exact match (EM) with entity-driven answer set expansion to expand gold answer set to include diverse surface forms. |
| Outcome: | The proposed method outperforms traditional evaluation methods while offering the benefits of high interpretability and reduced environmental harm. |
Don’t Judge Code by Its Cover: Exploring Biases in LLM Judges for Code Evaluation (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly used as evaluators for code evaluation tasks . however, whether they can handle superficial variations remains unclear . |
| Approach: | They define six types of potential biases in code evaluation and reveal their impact on LLM judges. |
| Outcome: | The proposed method can be used to evaluate semantically equivalent code with superficial variations without reference implementations. |
Fooling the LVLM Judges: Visual Biases in LVLM-Based Evaluation (2025.emnlp-main)
Copied to clipboard
| Challenge: | a recent study has found that large vision–language models are vulnerable to visual biases that inflate scores without altering semantic content. |
| Approach: | They propose a novel meta-evaluation benchmark that exhibits diverse score distributions. |
| Outcome: | The proposed model exhibits vulnerability across all domains, and combines multiple biases amplifies their effects, and pairwise evaluations are similarly susceptible. |
VLind-Bench: Measuring Language Priors in Large Vision-Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large Vision-Language Models suffer from a problem known as language prior . such language priors can lead to undesirable biases and hallucinations when dealing with images that are out of distribution. |
| Approach: | They propose a benchmark to measure the language priors of Large Vision-Language Models. |
| Outcome: | The proposed benchmark is the first specifically designed to measure the language priors, or blindness, of LVLMs. |
Mitigating Hallucinations in Large Vision-Language Models via Summary-Guided Decoding (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) generate detailed and coherent responses from visual inputs but are prone to generate hallucinations due to an over-reliance on language priors. |
| Approach: | They propose a method that reduces the text context and controls only the image-related POS tokens to maintain text quality by reducing the text contextualization. |
| Outcome: | The proposed method achieves state-of-the-art performance on object hallucination benchmarks and achieves Pareto optimality among the existing methods. |
Generating Diverse Hypotheses for Inductive Reasoning (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent studies suggest that large language models (LLMs) can engage in inductive reasoning by sampling multiple hypotheses about the rules and selecting the one that best explains the observations. |
| Approach: | They propose to increase the temperature parameter to enhance diversity by sampling multiple hypotheses and selecting the one that best explains the observations. |
| Outcome: | The proposed method improves diversity while maintaining text quality while increasing temperature. |
Asking Clarification Questions to Handle Ambiguity in Open-Domain QA (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Ambiguous questions persist in open-domain question answering because formulating a precise question with a unique answer is often challenging. |
| Approach: | They propose to ask a clarification question where the user’s response will help identify the interpretation that best aligns with the user's intention. |
| Outcome: | The proposed approach achieves F1 of 61.3, 25.1, and 40.5 on the three tasks, demonstrating the need for further improvements while providing competitive baselines for future work. |
Are LLM-Judges Robust to Expressions of Uncertainty? Investigating the effect of Epistemic Markers on LLM-based Evaluation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing efforts to train large language models to generate outputs containing epistemic markers have been largely overlooked. |
| Approach: | They propose a benchmark to assess the robustness of LLM-judges to epistemic markers. |
| Outcome: | EMBER benchmarks show that LLM-judges lack robustness in presence of epistemic markers . EMber QA (2,000 instances) and IF (2823 instances) are used to evaluate outputs containing epistemological markers. |
Can You Trick the Grader? Adversarial Persuasion of LLM Judges (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly used as automated evaluators in practical settings . |
| Approach: | a study by the university of california reveals that persuasive language can bias large language models when scoring mathematical reasoning tasks. |
| Outcome: | The proposed model can bias judges when scoring mathematical reasoning tasks . Consistency causes the most severe distortion, with Consistencies leading to 8% distortion . |