Papers by Philhoon Oh
CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Korean (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing benchmark datasets for Korean cultural and linguistic knowledge are derived from the English counterparts through translation, so they overlook cultural contexts. |
| Approach: | They propose to use Korean cultural and linguistic intelligence to assess Korean model performance by providing fine-grained annotations of cultural and cultural knowledge. |
| Outcome: | The proposed dataset includes 1,995 QA pairs and is based on 1,992 Korean exams and textbooks. |
Knowledge Corpus Error in Question Answering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work in open-domain question answering (QA) has explored generating context passages from large language models (LLMs) however, it is not well understood why generated passages can be more effective than retrieved ones. |
| Approach: | They propose to generate context passages from large language models by paraphrasing human-annotated gold context using LLMs to observe knowledge corpus error. |
| Outcome: | The proposed framework shows that paraphrasing human-annotated gold contexts improves performance over retrieval steps. |
Detrimental Contexts in Open-Domain Question Answering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Using the whole passages in QA datasets can improve model accuracy by 10% . |
| Approach: | They analyze how passages can have a detrimental effect on retrieve-then-read architectures used in question answering when evaluated on common question answering datasets. |
| Outcome: | The proposed model accuracy can be improved by 10% on two popular QA datasets by filtering out detrimental passages. |