Papers by Sungdong Kim
Generating Information-Seeking Conversations from Unlabeled Documents (2022.emnlp-main)
Copied to clipboard
| Challenge: | a novel framework for conversational question answering from unlabeled documents has been proposed . a large-scale dataset of synthetic conversations is available for use in real-world applications . |
| Approach: | They propose a framework for conversational question answering from unlabeled documents . they propose 'SimSeek' framework that simulates conversation from unlabelled documents based on two scenarios . |
| Outcome: | The proposed framework achieves state-of-the-art performance on a recent CQA benchmark, QuAC. |
KMMLU: Measuring Massive Multitask Language Understanding in Korean (2025.naacl-long)
Copied to clipboard
Guijin Son, Hanwool Lee, Sungdong Kim, Seungone Kim, Niklas Muennighoff, Taekyoon Choi, Cheonbok Park, Kang Min Yoo, Stella Biderman
| Challenge: | Recent models struggle to show performance over 60%, significantly below the pass mark of the source exams (80%), highlighting the room for improvement. |
| Approach: | They propose to use Korean exams to collect 35,030 questions from an expert-level multiple choice model to capture linguistic and cultural aspects of the Korean language. |
| Outcome: | The proposed benchmark is based on 35,030 questions from original Korean exams. |
Aligning Large Language Models by On-Policy Self-Judgment (2024.acl-long)
Copied to clipboard
| Challenge: | Existing approaches for aligning large language models with human preferences face a trade-off that requires a separate reward model for on-policy learning. |
| Approach: | They propose a new alignment framework that does on-policy learning and is parameter efficient . they propose Judge-augmented Supervised Fine-Tuning to train a single model to act as a policy and a judge. |
| Outcome: | The proposed framework outperforms baselines in preference benchmarks and rejecting sampling by itself improves performance without additional evaluator. |
Tree of Clarifications: Answering Ambiguous Questions with Retrieval-Augmented Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | In open-domain question answering, users often ask ambiguous questions (AQs) . one approach is to identify all possible interpretations of the AQ and generate a long-form answer addressing them all. |
| Approach: | They propose a framework that generates a long-form answer addressing all possible interpretations of an ambiguous question. |
| Outcome: | The proposed framework outperforms baselines on ASQA in a few-shot setup across metrics while surpassing fully-supervised baselines trained on the whole training set in terms of Disambig-F1 and Disambigo-ROUGE. |
Saving Dense Retriever from Shortcut Dependency in Conversational Search (2022.emnlp-main)
Copied to clipboard
| Challenge: | a retrieval shortcut in conversational search (CS) relies on partial history to retrieve relevant passages . naively trained dense retrievers heavily exploit the shortcut and perform poorly when asked to answer history-independent questions. |
| Approach: | They propose to exploit a retrieval shortcut in conversational search (CS) that allows models to only use partial history to retrieve relevant passages while disregarding the latest question. |
| Outcome: | The proposed model outperforms the previous state-of-the-art model by 11.0 on QReCC. |
Efficient Dialogue State Tracking by Selectively Overwriting Memory (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work in dialogue state tracking (DST) is inefficient in that it predicts dialogue state at every turn from scratch. |
| Approach: | They propose a method that selectively overwrites memory for dialogue state tracking by predicting dialogue state on each memory slot and overwriting it with new values. |
| Outcome: | The proposed model achieves state-of-the-art joint goal accuracy with 51.72% in MultiWOZ 2.0 and 53.01% in MultiWoz 2.1 in an open vocabulary-based DST setting. |
Generative Prompt Internalization (2025.naacl-long)
Copied to clipboard
| Challenge: | Prompts used in large language model based applications are often fixed and lengthy, leading to significant computational overhead. |
| Approach: | They propose a method that internalizes complex prompts using a joint training approach and a data synthesis technique that auto-collects conversational datasets by swapping roles of agent and environment. |
| Outcome: | The proposed method internalizes complex prompts across agent-based applications and generates the content along with reasons for why it should change accordingly. |
On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Model (2022.naacl-main)
Copied to clipboard
Seongjin Shin, Sang-Woo Lee, Hwijeen Ahn, Sungdong Kim, HyoungSeok Kim, Boseop Kim, Kyunghyun Cho, Gichang Lee, Woomyoung Park, Jung-Woo Ha, Nako Sung
| Challenge: | Recent studies on large-scale in-context language models have reported successful in-const zero- and few-shot learning ability. |
| Approach: | They investigate the effects of the pretraining corpus on in-context learning in a Korean-centric model. |
| Outcome: | The study shows that pretraining corpus size does not determine in-context learning ability . the findings suggest that in-constext learning is not always competitive . |
NeuralWOZ: Learning to Collect Task-Oriented Dialogue via Model-Based Simulation (2021.acl-long)
Copied to clipboard
| Challenge: | NeuralWOZ generates dialogues from user’s goal instructions and system’s API call results. |
| Approach: | They propose a framework that uses model-based dialogue simulation to generate dialogues from user’s goal instructions and system’s API call results. |
| Outcome: | The proposed framework achieves 4.4% point joint goal accuracy on average across domains and 5.7% point of zero-shot coverage against the MultiWOZ 2.1 dataset. |
Rethinking the Role of Proxy Rewards in Language Model Alignment (2024.emnlp-main)
Copied to clipboard
| Challenge: | Typically, the human feedback is used to train a proxy reward model (RM), and a policy model is optimized over the reward signal from the RM using RL. |
| Approach: | They aim to replicate the ground truth (gold) reward signal by achieving a monotonic relationship between the proxy and gold reward signals after training the model using the proxy reward in reinforcement learning (RL). |
| Outcome: | The proposed model shows competitive performances with strong open-source RMs in alignment benchmarks. |
Keep Me Updated! Memory Management in Long-term Conversations (2022.findings-emnlp)
Copied to clipboard
Sanghwan Bae, Donghyun Kwak, Soyoung Kang, Min Young Lee, Sungdong Kim, Yuin Jeong, Hyeri Kim, Sang-Woo Lee, Woomyoung Park, Nako Sung
| Challenge: | Existing studies do not deal with cases where memorized information is outdated, which may cause confusion in later conversations. |
| Approach: | They propose a task where bots keep track of and bring up the latest information about users while conversing through multiple sessions. |
| Outcome: | The proposed method outperforms baselines that leave the stored memory unchanged in terms of engagingness and humanness, and a larger performance gap in the later sessions. |
LangBridge: Multilingual Reasoning Without Multilingual Supervision (2024.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to adapt language models for multilingual reasoning tasks require multilingual supervision. |
| Approach: | They propose a zero-shot approach to adapt language models for multilingual reasoning tasks without multilingual supervision by bridging two models by introducing minimal trainable parameters between them. |
| Outcome: | The proposed approach significantly improves multilingual reasoning capabilities on low-resource languages. |
Can Language Models be Biomedical Knowledge Bases? (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have focused on probing LMs in the general domain but little attention has been given to whether they can be used as domain knowledge bases. |
| Approach: | They propose to use 49K biomedical factual knowledge triples to probe LMs for biomedically . they find that biomedic LM can achieve up to 18.51% Acc@5 on retrieving biomedcial knowledge. |
| Outcome: | The proposed biomedical factual knowledge probing benchmark achieves 18.51% Acc@5 on biomedically-relevant knowledge retrieval. |
Ask Optimal Questions: Aligning Large Language Models with Retriever’s Preference in Conversation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to perform conversational search are sub-optimal due to the limited ability to incorporate signals from the retrieval results. |
| Approach: | They propose to optimize a language model for reformulating search queries in line with retrievers’ preferences by combining a large-scale dataset with Retrievers’ Feedback. |
| Outcome: | The proposed framework outperforms existing methods on two benchmarks and surpasses the state-of-the-art methods. |
Gradient Ascent Post-training Enhances Language Model Generalization (2023.acl-short)
Copied to clipboard
| Challenge: | Recent studies show that language models can perform diverse downstream NLP tasks in a zero-shot manner without any gradient updates. |
| Approach: | They show that updating pretrained LMs with Gradient Ascent Post-training enhances zero-shot generalization capabilities without any task-specific fine-tuning. |
| Outcome: | The proposed method can be comparable to 2-3x times larger LMs across 12 different NLP tasks. |
Building a Role Specified Open-Domain Dialogue System Leveraging Large-Scale Language Models (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent large-scale language models have produced human-like responses in open-domain dialogue systems. |
| Approach: | They propose a framework for imposing roles on open-domain dialogue systems . they use few-shot learning to build a Korean dialogue dataset from scratch . |
| Outcome: | The proposed framework meets role specifications while maintaining conversational abilities. |
Aligning Large Language Models through Synthetic Feedback (2023.emnlp-main)
Copied to clipboard
| Challenge: | Currently, alignment learning requires significant human demonstrations and feedback from proprietary LLMs such as ChatGPT. |
| Approach: | They propose a framework that uses synthetic feedback to align large language models to human values without extensive human annotations and proprietary LLMs. |
| Outcome: | The proposed model outperforms open-source models on human-annotated demonstrations in alignment benchmarks. |