Papers by Miyoung Ko
Ranking Paragraphs for Improving Answer Recall in Open-Domain Question Answering (D18-1)
Copied to clipboard
| Challenge: | Recent work has combined open-domain question answering with machine comprehension models to find answers in a large knowledge source. |
| Approach: | They propose a machine comprehension model that ranks paragraphs of retrieved documents for a higher answer recall with less noise. |
| Outcome: | The proposed model improves on four open-domain QA datasets by 7.8% on average. |
Hierarchical Deconstruction of LLM Reasoning: A Graph-Based Framework for Analyzing Knowledge Utilization (2024.emnlp-main)
Copied to clipboard
| Challenge: | Despite advances in large language models, how they use their knowledge for reasoning is not yet well understood. |
| Approach: | They propose a method that deconstructs complex real-world questions into a graph . they quantify forward discrepancy, a discrepany in LLM performance on simpler sub-problems . |
| Outcome: | The proposed method shows that smaller models exhibit more discrepancies than larger models . it also shows that guiding models from simpler to complex questions improves performance . |
ClaimDiff: Comparing and Contrasting Claims on Contentious Issues (2023.findings-acl)
Copied to clipboard
| Challenge: | Using fact verification tasks, however, can not detect subtle differences in factually consistent claims, which might bias the readers. |
| Approach: | They propose a novel dataset that primarily focuses on comparing the nuance between claim pairs. |
| Outcome: | The proposed dataset shows that human-labeled 2,941 claim pairs are weaker than baselines, showing a 19% absolute gap with the baselines. |
The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models (2025.naacl-long)
Copied to clipboard
Seungone Kim, Juyoung Suk, Ji Yong Cho, Shayne Longpre, Chaeeun Kim, Dongkeun Yoon, Guijin Son, Yejin Cho, Sheikh Shafayat, Jinheon Baek, Sue Hyun Park, Hyeonbin Hwang, Jinkyung Jo, Hyowon Cho, Haebin Shin, Seongyun Lee, Hanseok Oh, Noah Lee, Namgyu Ho, Se June Joo, Miyoung Ko, Yoonjoo Lee, Hyungjoo Chae, Jamin Shin, Joel Jang, Seonghyeon Ye, Bill Yuchen Lin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, Minjoon Seo
| Challenge: | a recent study evaluated language models using abstract evaluation criteria that lack the flexibility and granularity of human assessment. |
| Approach: | They propose a benchmark to evaluate nine distinct language models' capabilities . they use instance-specific evaluation criteria to mirror human evaluation . |
| Outcome: | The proposed benchmark evaluates nine distinct capabilities of language models across 77 tasks. |
KTRL+F: Knowledge-Augmented In-Document Search (2024.naacl-long)
Copied to clipboard
| Challenge: | KTRL+F is a knowledge-augmented in-document search that requires real-time identification of all semantic targets within a document with the awareness of external sources through a single natural query. |
| Approach: | They propose a knowledge-augmented in-document search that requires real-time identification of all semantic targets within a document with the awareness of external sources through a single natural query. |
| Outcome: | The proposed model reduces time for searching with less queries and reduced extra visits to other sources for collecting evidence. |
Look at the First Sentence: Position Bias in Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Extractive question answering models are trained to predict start and end positions of answers . recent QA models outperform humans in some datasets due to their simplicity and effectiveness. |
| Approach: | They propose to use prior distribution of answer positions as a bias model to reduce position bias. |
| Outcome: | The proposed model outperforms BERT from 37.48% to 81.64% when trained on a biased SQUAD dataset. |