Papers by Minju Song
ETHIC: Evaluating Large Language Models on Long-Context Tasks with High Information Coverage (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing evaluation methods do not assess whether large language models fully utilize contextual information. |
| Approach: | They introduce a new metric to assess LLMs' ability to fully utilize contextual information. |
| Outcome: | The proposed benchmark comprises 1,986 test instances spanning four long-context tasks with high IC scores in the domains of books, debates, medicine, and law. |
BotsTalk: Machine-sourced Framework for Automatic Curation of Large-scale Multi-skill Dialogue Datasets (2022.emnlp-main)
Copied to clipboard
| Challenge: | a number of largescale datasets targeting a specific conversational skill have recently become available. |
| Approach: | They propose a framework where multiple agents grounded to specific skills participate in a conversation to automatically annotate multi-skill dialogues. |
| Outcome: | The proposed framework can be used to build open-domain chatbots with diverse communicative skills. |
Assessing LLM Reasoning Steps via Principal Knowledge Grounding (2025.findings-emnlp)
Copied to clipboard
Hyeon Hwang, Yewon Cho, Chanwoong Yoon, Yein Park, Minju Song, Kyungjae Lee, Gangwoo Kim, Jaewoo Kang
| Challenge: | Step-by-step reasoning has become a standard approach for large language models to tackle complex tasks. |
| Approach: | They propose a framework that assesses the knowledge grounding of intermediate reasoning by using a large-scale repository of atomic knowledge essential for reasoning. |
| Outcome: | The evaluation suite identifies missing or misapplied knowledge elements and provides crucial insights for uncovering fundamental reasoning deficiencies in LLMs. |