Dial-In LLM: Human-Aligned LLM-in-the-loop Intent Clustering for Customer Service Dialogues (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing intent clustering methods rely on embedding distance metrics and neglect of underlying semantic structures. |
| Approach: | They propose an LLM-in-the-loop framework that integrates language understanding capabilities into conventional clustering algorithms. |
| Outcome: | The proposed framework outperforms baselines in Chinese and improves quality, cost efficiency and downstream applications. |
Similar Papers
Intent Detection and Discovery from User Logs via Deep Semi-Supervised Contrastive Clustering (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to intent detection rely on epoch wise clustering and classification based on labeled and unlabeled data. |
| Approach: | They propose an end-to-end deep contrastive clustering algorithm that jointly updates model parameters and cluster centers via supervised and self-supervised learning. |
| Outcome: | The proposed approach outperforms baselines on five public datasets and human-in-the-loop variant for practical deployment. |
Learning LLM Preference over Intra-Dialogue Pairs: A Framework for Utterance-level Understandings (2025.naacl-industry)
Copied to clipboard
Xuanqing Liu, Luyang Kong, Wei Niu, Afshin Khashei, Belinda Zeng, Steve Johnson, Jon Jay, Davor Golac, Matt Pope
| Challenge: | Large language models (LLMs) have demonstrated remarkable capabilities in handling complex dialogue tasks without requiring use case-specific fine-tuning. |
| Approach: | They propose a framework that combines the scalability of LLM-generated labels with the precision of human annotations to achieve higher speed and accuracy comparable to larger models. |
| Outcome: | The proposed framework significantly improves accuracy across utterance-level dialogue tasks, including sentiment detection (over 2%), dialogue act classification (over 1.5%), etc. |
Controllable Clustering with LLM-driven Embeddings (2025.emnlp-industry)
Copied to clipboard
Kerria Pang-Naylor, Shivani Manivasagan, Aitong Zhong, Mehak Garg, Nicholas Mondello, Blake Buckner, Jonathan P. Chang, Khyati Mahajan, Masoud Hashemi, Fabio Casati
| Challenge: | Unsupervised text clustering is unlikely to produce groupings that work across use cases . authors present techniques to effectively control text embeddings with minimal human input . |
| Approach: | They propose techniques to control text embeddings with minimal human input . they evaluate clustering performance for datasets with multiple independent labels . |
| Outcome: | The proposed techniques improve clustering for one perspective or use case, but at a tradeoff in performance for another use case. |
Auto-Dialabel: Labeling Dialogue Data with Unsupervised Learning (D18-1)
Copied to clipboard
| Challenge: | Existing dialog datasets rely on human labeling, which is expensive, limited in size, and in low coverage. |
| Approach: | They propose a framework to automatically cluster dialogue intents and slots . they collect context features, leverage an autoencoder for feature assembly, and adapt a dynamic hierarchical clustering method for intent and slot labeling. |
| Outcome: | The proposed framework can promote human labeling cost to a great extent and achieve good intent clustering accuracy (84.1%) it also provides reasonable and instructive slot labeling results. |
A User-Centric Multi-Intent Benchmark for Evaluating Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks focus on specific predefined model abilities, such as world knowledge, reasoning, etc., making it difficult for users to determine which LLM best suits their particular needs. |
| Approach: | They propose to evaluate large language models from a user-centric perspective and use real-world use cases to identify their effectiveness under distinct intents. |
| Outcome: | The proposed benchmarks achieve a correlation between human preference and the user-reported scenarios and human intents. |
LLMs Enable Bag-of-Texts Representations for Short-Text Clustering (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for short text clustering require labeling and no embeddings optimization. |
| Approach: | They propose a training-free method for unsupervised short text clustering that relies less on careful selection of embedders than other methods. |
| Outcome: | The proposed method achieves comparable or superior results to state-of-the-art methods, but without embeddings optimization or prior knowledge of clusters or labels. |
Large Language Models Know Your Contextual Search Intent: A Prompting Framework for Conversational Search (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for understanding users’ contextual search intent show unsatisfactory effectiveness and robustness to handle real conversational search scenarios. |
| Approach: | They propose to use large language models to generate multiple query rewrites and hypothetical responses and to aggregate them into an integrated representation that can robustly represent the user’s real contextual search intent. |
| Outcome: | The proposed framework can generate multiple query rewrites and hypothetical responses and can be used to represent the user’s real contextual search intent. |
Do LLMs Understand Dialogues? A Case Study on Dialogue Acts (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown remarkable performance on many unseen tasks in a zero-shot setting. |
| Approach: | They propose to identify three key pre-tasks essential for accurate DA prediction: Turn Management, Communicative Function Identification, and Dialogue Structure Prediction. |
| Outcome: | The proposed model fails to outperform basic rule-based tasks on three key pre-tasks, and the results suggest that the model is flawed. |
LLM-Guided Semantic Relational Reasoning for Multimodal Intent Recognition (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for understanding intents from multimodal signals exhibit limitations in their modality-level reliance, constraining relational reasoning over fine-grained semantics for complex intent understanding. |
| Approach: | They propose a method that harnesses the expansive knowledge of large language models to establish semantic foundations that boost smaller models’ relational reasoning performance. |
| Outcome: | The proposed method outperforms state-of-the-art methods on multimodal intent and dialogue act recognition tasks and shows consistent performance gains across diverse semantic understanding scenarios. |
Scaling Intent Understanding: A Framework for Classification with Clarification using Lightweight LLMs (2026.eacl-industry)
Copied to clipboard
| Challenge: | Proprietary large-language models (LLMs) assign intents to user utterances without addressing ambiguity. |
| Approach: | They propose a domain-agnostic framework that equips open-source LLMs with the ability to perform intent classification and generate clarification questions in case of ambiguity. |
| Outcome: | The proposed framework performs intent classification and generates clarification questions in case of ambiguity. |