Papers by Byungkyu Kang
Unsupervised Improvement of Factual Knowledge in Language Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Masked language modeling (MLM) is often dominated by high-frequency words that are sub-optimal for learning factual knowledge. |
| Approach: | They propose an approach that forces the model to prioritize informative words in a fully unsupervised way. |
| Outcome: | The proposed approach significantly improves the performance of pretrained language models on factual recall, question answering, sentiment analysis, and natural language inference in a closed-book setting. |
Mitigating Hallucination in Fictional Character Role-Play (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Influence of parametric knowledge of large language models (LLMs) often causes role-playing characters to act out of character and hallucinate about things outside the scope of their knowledge. |
| Approach: | They propose a method that modulates the influence of parametric knowledge using a pre-calibrated confidence threshold to mitigate hallucination in fictional character role-play. |
| Outcome: | The proposed method reduces the factual accuracy of generated responses by 18% for adversarial questions and 44% in temporal hallucination for time-sensitive interviews. |