Papers by Joongbo Shin
FQ-Eval: Building Evaluation Dataset for User-centered Follow-up Question Generation (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing studies focus on enhancing information-seeking or topical relevance, often missing how follow-up questions could satisfy users’ intrinsic needs and conversational goals. |
| Approach: | They propose a user-centered evaluation dataset for assessing follow-up question generation in chat-LLM services that incorporates realistic chat-llm usage scenarios and five distinct human-aligned criteria. |
| Outcome: | The proposed model captures human-aligned criteria for the evaluation of various models and services. |
Deep Exploration of Cross-Lingual Zero-Shot Generalization in Instruction Tuning (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on instruction tuning to show cross-lingual generalization . a novel non-English meta-dataset is used to study instruction tuning . |
| Approach: | They perform instruction tuning individually for two distinct language meta-datasets and assess the performance on unseen tasks in a non-English language. |
| Outcome: | The proposed model outperforms baseline training in English and Korean by 20.7% and 13.6%. |
FaithUn: Toward Faithful Forgetting in Language Models by Investigating the Interconnectedness of Knowledge (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for unlearning undesirable knowledge have overlooked complexity and interconnectedness of knowledge, authors say . previous studies have neglected the complex nature of knowledge and neglected its internal dependencies. |
| Approach: | They propose a new concept called superficial unlearning to evaluate faithfulness of unlearning in knowledge QA settings. |
| Outcome: | The proposed method shows significant effectiveness in real-world knowledge QA settings. |
KPQA: A Metric for Generative Question Answering Using Keyphrase Weights (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing n-gram similarity metrics fail to discriminate the incorrect answers due to the free-form of the answer. |
| Approach: | They propose a new metric that assigns different weights to each token via keyphrase prediction to judge the correctness of GenQA. |
| Outcome: | The proposed metric has a significantly higher correlation with human judgments than existing metrics in various datasets. |
Fast and Accurate Deep Bidirectional Language Representations for Unsupervised Learning (2020.acl-main)
Copied to clipboard
| Challenge: | Existing deep bidirectional language models are limited by repetitive inferences on unsupervised tasks for the computation of contextual language representations. |
| Approach: | They propose a deep bidirectional language model called a Transformer-based Text Autoencoder (T-TA) it computes contextual language representations without repetition and shows competitive or even better accuracies than BERT . |
| Outcome: | The proposed model performs six times faster on a reranking task and twelve times faster in a semantic similarity task. |
BREAK: Breaking the Dialogue State Tracking Barrier with Beam Search and Re-ranking (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for dialogue state tracking still have a JGA of 60% on MultiWOZ 2.1 . break framework provides a simple yet effective way to generate dialogue state candidates . |
| Approach: | They propose a framework that generates k-best dialogue state candidates with beam search and re-ranks them to select the correct dialogue state. |
| Outcome: | The proposed framework pushes the joint goal accuracy to 80-90% on MultiWOZ 2.1-2.4. |
Dynamic Label Name Refinement for Few-Shot Dialogue Intent Classification (2025.acl-short)
Copied to clipboard
| Challenge: | Current intent classification systems face significant challenges due to the vast number of possible intents and significant semantic overlap among similar intent classes. |
| Approach: | They propose a dynamic label refinement method that retrieves relevant examples for a test input and leverages a large language model to dynamically refine intent labels based on semantic understanding. |
| Outcome: | The proposed method resolves confusion between semantically similar intents and generates more interpretable intent labels. |
Learning to Rank Question-Answer Pairs Using Hierarchical Recurrent Encoder with Latent Topic Clustering (N18-1)
Copied to clipboard
| Challenge: | Existing models for sentence pair ranking are based on hierarchical recurrent neural network and latent topic clustering module. |
| Approach: | They propose a hierarchical recurrent neural network and latent topic clustering module to adapt a recursive hierarchic neural network to rank candidate answers. |
| Outcome: | The proposed model shows small performance degradations in longer text comprehension compared to current models which suffer from it. |
TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models (2022.emnlp-main)
Copied to clipboard
Joel Jang, Seonghyeon Ye, Changho Lee, Sohee Yang, Joongbo Shin, Janghoon Han, Gyeonghun Kim, Minjoon Seo
| Challenge: | Language Models (LMs) become outdated as the world changes, a phenomenon called temporal misalignment. |
| Approach: | They propose a lifelong benchmark that utilizes the difference between consecutive snapshots of English Wikipedia and English Wikidata for training and evaluation. |
| Outcome: | The proposed benchmark can be trained on the difference between consecutive snapshots of English Wikipedia and English Wikidata for training and evaluation. |
Injecting Comparison Skills in Task-Oriented Dialogue Systems for Database Search Results Disambiguation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing task-oriented dialogue systems handle ambiguity in database search results by randomly selecting one or few results and presenting their names to the user. |
| Approach: | They propose a task that compares properties of multiple database search results . they use a dataset to collect high-quality dialogue data and an augmented version of the SGD dataset . |
| Outcome: | The proposed task compares properties of two entities in a trade-off form based on user preferences . the proposed dataset and code will be publicized . |