Papers by Sungjin Lee
Learning Slice-Aware Representations with Mixture of Attentions (2021.findings-acl)
Copied to clipboard
| Challenge: | Real-world machine learning systems are achieving excellent performance in terms of coarse-grained metrics like overall accuracy and F-1 score. |
| Approach: | They extend slice-based learning (SBL) with a mixture of attentions to learn slice-aware dual attentive representations. |
| Outcome: | The proposed approach outperforms the baseline method and the original SBL approach on monitored slices with two natural language understanding tasks. |
A Scalable Framework for Learning From Implicit User Feedback to Improve Natural Language Understanding in Large-Scale Conversational AI Systems (2021.emnlp-main)
Copied to clipboard
Sunghyun Park, Han Li, Ameen Patel, Sidharth Mudgal, Sungjin Lee, Young-Bum Kim, Spyros Matsoukas, Ruhi Sarikaya
| Challenge: | Existing methods to improve NLU are laborintensive and expensive. |
| Approach: | They propose a scalable and automatic approach to improving NLU in a large-scale conversational AI system by leveraging implicit user feedback. |
| Outcome: | The proposed framework improves NLU in a large-scale conversational AI system across 10 domains. |
Scalable and Robust Self-Learning for Skill Routing in Large-Scale Conversational AI Systems (2022.naacl-industry)
Copied to clipboard
| Challenge: | Existing methods to enable skill routing do not scale in terms of the number of skills and skill on-boarding. |
| Approach: | They propose a model-based approach to enable natural conversation by allowing frequent policy updates . they propose an annotation-based system, rule-based model, and bandit-based learning . |
| Outcome: | The proposed method is scalable and cost-effective, the authors show . they show that it can improve the user experience without abrupt policy changes . |
AugNLG: Few-shot Natural Language Generation using Self-trained Data Augmentation (2021.acl-long)
Copied to clipboard
| Challenge: | Large-scale conversational systems typically generate unnatural, robotic responses using template-based approaches. |
| Approach: | They propose a data augmentation approach that combines a self-trained neural retrieval model with a few-shot learned NLU model to automatically create MR-to-Text data from open-domain texts. |
| Outcome: | The proposed approach outperforms the state-of-the-art methods on the FewshotWOZ data in both BLEU and Slot Error Rate. |
PENTATRON: PErsonalized coNText-Aware Transformer for Retrieval-based cOnversational uNderstanding (2022.emnlp-industry)
Copied to clipboard
| Challenge: | In a large fraction of the global traffic from smart digital assistants, frictions in dialogues may be attributed to incorrect understanding of the entities in a user's query due to factors including ambiguous mentions, mispronunciation, background noise and faulty on-device signal processing. |
| Approach: | They propose a parametric transformer-based language model to learn patterns from in-session customer-device interactions coupled with a non-parametric personalized entity index to compute the correct query. |
| Outcome: | The proposed system improves on the existing system and shows that it can learn the correct query from in-session customer-device interactions. |
Cluster-Guided Label Generation in Extreme Multi-Label Classification (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing classification-based models are poorly per-form for tail labels and ignore semantic relations among labels. |
| Approach: | They propose to guide label generation using label cluster information to hierarchically generate lower-level labels. |
| Outcome: | The proposed model outperforms classification and generation baselines on tail labels and improves in four popular XMC benchmarks. |
Explaining Dialogue Evaluation Metrics using Adversarial Behavioral Analysis (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing frameworks for dialogue model evaluation are lacking to investigate these biases . a number of dialogue metrics are biased and can cause unforeseen problems . |
| Approach: | They propose an adversarial test-suite which generates problematic variations of various dialogue aspects using automatic heuristics. |
| Outcome: | The proposed test-suite generates problematic variations of various dialogue aspects using automatic heuristics. |
Large-scale Lifelong Learning of In-context Instructions and How to Tackle It (2023.acl-long)
Copied to clipboard
| Challenge: | In-context instruction learning is a method to improve the target PLM’s instance- and task-level generalization performance as it observes more tasks. |
| Approach: | They propose to fine-tune a Pre-trained Language Model (PLM) on a set of tasks with in-context instructions and to extend this property to a scenario in which tasks are fed to the target PLM in a sequential manner. |
| Outcome: | The proposed method achieves noticeable improvements in both types of generalization, nearly reaching the upper bound performance obtained through joint training. |
Generating a Common Question from Multiple Documents using Multi-source Encoder-Decoder Models (D19-56)
Copied to clipboard
| Challenge: | Ambiguous user queries can result in multiple topics being retrieved from search engines. |
| Approach: | They propose a task of generating a common question from multiple documents by training an RNN-based single encoder-decoder generator from document pairs and then a model that aggregates these word distributions to generate a question. |
| Outcome: | The proposed model significantly outperforms existing models when evaluated using automated metrics and human judgments on the MS-MARCO-QA dataset. |
Self-Supervised Contrastive Learning for Efficient User Satisfaction Prediction in Conversational Agents (2021.naacl-main)
Copied to clipboard
| Challenge: | End-to-end deep learning methods that focus on user satisfaction are challenging due to the required annotation costs and turnaround times. |
| Approach: | They propose a self-supervised contrastive learning approach that leverages the pool of unlabeled data to learn user-agent interactions. |
| Outcome: | The proposed approach reduces the required number of annotations while improving generalization on unseen skills. |
FaVe: Factored and Verified Search Rationale for Long-form Answer (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing solutions for long-form question-answering (LFQA) use chain-of-thought (CoT) with retrieval-augmented generation (RAG). |
| Approach: | They propose to integrate chain-of-thought (CoQ) with retrieval-augmented generation to improve answer comprehensiveness and verifiability. |
| Outcome: | The proposed approach outperforms ChatGPT baselines while maintaining efficiency. |
Data-Efficient Goal-Oriented Conversation with Dialogue Knowledge Transfer Networks (D19-1)
Copied to clipboard
| Challenge: | Goal-oriented dialogue systems are now being widely adopted in industry where it is of key importance to maintain a rapid prototyping cycle for new products and domains. |
| Approach: | They propose a data-driven approach to goal-oriented dialogue generation which only uses a few example dialogues, none of which has to be annotated. |
| Outcome: | The proposed approach significantly improves upon baseline models and over the previous state-of-the-art model, ZSDG. |
FreeTalky: Don’t Be Afraid! Conversations Made Easier by a Humanoid Robot using Persona-based Dialogue (2022.lrec-1)
Copied to clipboard
| Challenge: | FreeTalky is a deep learning-based foreign language learning platform for people who experience anxiety dealing with foreign languages. |
| Approach: | They propose a deep learning-based foreign language learning platform called FreeTalky . it employs a humanoid robot NAO and various deep learning models . |
| Outcome: | The proposed system provides personalized learning based on persona dialogue and grammar error correction, and also helps alleviate xenoglossophobia by replacing the real human in the conversation with a NAO robot, through human evaluation. |
Constrained Policy Optimization for Controlled Self-Learning in Conversational AI Systems (2023.acl-industry)
Copied to clipboard
| Challenge: | Recent self-learning methods based on user satisfaction metrics and contextual bandits have shown promising results to enable consistent improvements in conversational AI systems. |
| Approach: | They propose a meta-gradient learning approach that adjusts constraint violation penalty terms adaptively through a user-defined meta objective that encourages balanced constraint satisfaction across domains. |
| Outcome: | The proposed framework supports fine-grained exploration targets for individual domains via user-defined constraints. |
Structuring Latent Spaces for Stylized Response Generation (D19-1)
Copied to clipboard
| Challenge: | Existing methods for generating responses in a targeted style are limited by the lack of parallel data. |
| Approach: | They propose a method that bridges conversation modeling and non-parallel style transfer by sharing a structured latent space. |
| Outcome: | The proposed system generates responses of the targeted style and outperforms baselines without sacrificing appropriateness. |
Guided Dialogue Policy Learning without Adversarial Learning in the Loop (2020.findings-emnlp)
Copied to clipboard
Ziming Li, Sungjin Lee, Baolin Peng, Jinchao Li, Julia Kiseleva, Maarten de Rijke, Shahin Shayandeh, Jianfeng Gao
| Challenge: | Reinforcement learning methods suffer from sparse and unstable reward signals . alternating training of dialogue agent and reward model can get stuck in local optima . |
| Approach: | They propose to decompose adversarial training into two steps to improve dialogue policy learning. |
| Outcome: | The proposed method achieves remarkable task success rate using both on-policy and off-poly reinforcement learning methods. |
Selective In-Context Data Augmentation for Intent Detection using Pointwise V-Information (2023.eacl-main)
Copied to clipboard
Yen-Ting Lin, Alexandros Papangelis, Seokhwan Kim, Sungjin Lee, Devamanyu Hazarika, Mahdi Namazifar, Di Jin, Yang Liu, Dilek Hakkani-Tur
| Challenge: | Intent detection is a fundamental element in task-oriented dialogue systems, usually occurring within the Natural Language Understanding component. |
| Approach: | They propose an in-context data augmentation approach that fine-tunes a pre-trained language model and synthesizes new datapoints that correspond to given intents. |
| Outcome: | The proposed method produces training data that achieves state-of-the-art on three challenging intent detection datasets and performs on par with the state- of-the art in full-shot settings. |
ConvLab: Multi-Domain End-to-End Dialog System Platform (P19-3)
Copied to clipboard
Sungjin Lee, Qi Zhu, Ryuichi Takanobu, Zheng Zhang, Yaoqin Zhang, Xiang Li, Jinchao Li, Baolin Peng, Xiujun Li, Minlie Huang, Jianfeng Gao
| Challenge: | ConvLab is an open-source multi-domain end-to-end dialog system platform . it allows researchers to quickly set up experiments with reusable components and compare a large set of different approaches in common environments. |
| Approach: | They propose to use an open-source multi-domain end-to-end dialog system platform to train and evaluate dialog bots in common environments. |
| Outcome: | The proposed system enables researchers to quickly set up experiments with reusable components and compare a large set of different approaches in common environments. |
Jointly Optimizing Diversity and Relevance in Neural Response Generation (N19-1)
Copied to clipboard
| Challenge: | Recent neural conversation models often generate bland and generic responses . however, the improvement often comes at the cost of decreased relevance . |
| Approach: | They propose a spacefusion model to jointly optimize diversity and relevance that fuses the latent space of a sequence-to-sequence model and that of an autoencoder model by leveraging novel regularization terms. |
| Outcome: | The proposed model improves diversity and relevance compared to baselines in both diversity and diversity. |
Open World Classification with Adaptive Negative Samples (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing models with no effective open category data during training are limited by the lack of effective open categories data during the training stage. |
| Approach: | They propose an approach to generate effective open category samples in the training stage and without requiring prior knowledge or external datasets. |
| Outcome: | The proposed approach generates effective synthetic open category samples in the training stage and without requiring any prior knowledge or external datasets. |