Reliable and Interpretable Drift Detection in Streams of Short Texts (2023.acl-industry)
Copied to clipboard
| Challenge: | Data drift is a key factor leading to model performance degradation over time. |
| Approach: | They propose a framework for reliable model-agnostic change-point detection and interpretation in large task-oriented dialog systems. |
| Outcome: | The proposed framework is effective in multiple customer deployments. |
Similar Papers
Targeting the Needle, Ignoring the Haystack: Anchoring Crucial Cues for Evolving Scam Call Detection via an LLM-Assisted Classifier (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for fraud detection on online service platforms often fail to generalize due to the scarcity of labeled data and the continuous evolution of conversational contexts. |
| Approach: | They propose a framework that anchors detection on Semantic Primitives . they prioritize stable evidence over conversational noise to ensure a verifiable fraud tactic . |
| Outcome: | The proposed framework achieves superior robustness and efficiency compared to baselines . it prioritizes stable evidence over diverse conversational noise . |
Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems (P18-1)
Copied to clipboard
| Challenge: | End-to-end task-oriented dialog systems often suffer from the challenge of incorporating knowledge bases. |
| Approach: | They propose a novel yet simple end-to-end differentiable model called memory-tosequence to address this issue. |
| Outcome: | The proposed model can be trained faster and achieve state-of-the-art performance on three different task-oriented dialog datasets. |
Actionable Interpretability for Churn Classification: A Text Bottleneck Model Case Study at a Major Telecom Provider (2026.acl-industry)
Copied to clipboard
| Challenge: | Managing customer churn is vital for subscription-based businesses . large language models (LLMs) can automate the classification of chursn-intent at scale . lack of transparency forces a difficult choice between automated systems and manual review . |
| Approach: | They propose to use text bottleneck models to classify customer churn in subscription-based businesses . they show that the model can be used to bridge the perceived trade-off between interpretability andpredictive performance . |
| Outcome: | The proposed model performs competitively with black-box baselines and an interactive dashboard. |
Decoupled Reasoning with Implicit Fact Tokens (DRIFT): A Dual-Model Framework for Efficient Long-Context Inference (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing solutions to integrate extensive, dynamic knowledge into Large Language Models (LLMs) are constrained by finite context windows, retriever noise, or the risk of catastrophic forgetting. |
| Approach: | They propose a dual-model architecture that explicitly decouples knowledge extraction from the reasoning process by compressing document chunks into implicit fact tokens conditioned on the query. |
| Outcome: | The proposed architecture significantly outperforms strong baselines among comparably sized models on long-context tasks while maintaining inference accuracy. |
Revitalizing Black-Box Interpretability: Actionable Interpretability for LLMs via Proxy Models (2026.acl-long)
Copied to clipboard
| Challenge: | Applying model-agnostic explanations to Large Language Models is hindered by prohibitive computational costs rendering them dormant for real-world applications. |
| Approach: | They propose a budget-friendly proxy framework that leverages efficient models to approximate the decision boundaries of expensive Large Language Models. |
| Outcome: | The proposed framework achieves over 90% fidelity with only 9.5% of the oracle’s cost and is open-source to facilitate future research. |
Navigating the Unknown: Intent Classification and Out-of-Distribution Detection Using Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Out-of-Distribution (OOD) detection requires great generalization capability . |
| Approach: | They propose a method that is cost-efficient, high-performing, highly robust and versatile enough to be used with smaller LLMs without sacrificing performance. |
| Outcome: | The proposed method is cost-efficient, high-performing, robust, and versatile enough to be used with smaller LLMs without sacrificing performance. |
When and Why a Model Fails? A Human-in-the-loop Error Detection Framework for Sentiment Analysis (2021.naacl-industry)
Copied to clipboard
| Challenge: | Existing methods for sentiment analysis are difficult to assess for erroneous predictions that might exist prior to deployment. |
| Approach: | They propose a framework for error detection based on explainable features that can detect erroneous model predictions on unseen data with high precision. |
| Outcome: | The proposed framework detects erroneous model predictions on unseen data with high precision, given limited human-in-the-loop intervention, and can be deployed on unselected data with a high accuracy. |
Interpretable Traces, Unexpected Outcomes: Investigating the Disconnect in Trace-Based Knowledge Distillation (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in reasoning-oriented Large Language Models have been driven by the introduction of Chain-of-Thought (CoT) traces. |
| Approach: | They propose to use CoT traces to guide model inference and serve as supervision signals for Knowledge Distillation to improve smaller models. |
| Outcome: | The proposed model is based on a rule-based problem decomposition method and is valid for both semantic correctness and interpretability to the end user. |
Path Drift in Large Reasoning Models: How First-Person Commitments Override Safety (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on prompt injection and jailbreak attacks primarily target the surface structure of input prompts. |
| Approach: | They propose a three-stage approach to mitigate the risk of Long-CoT reasoning drift . they propose 'path-level defense' strategy that incorporates role attribution correction and metacognitive reflection . |
| Outcome: | The proposed framework reduces refusal rates and ethical evaporation, while ethical escalation and layered disclaimers progressively steer models toward unsafe completions. |
Improving Large-Scale Conversational Assistants using Model Interpretation based Training Sample Selection (2022.emnlp-industry)
Copied to clipboard
Stefan Schroedl, Manoj Kumar, Kiana Hajebi, Morteza Ziyadi, Sriram Venkatapathy, Anil Ramakrishna, Rahul Gupta, Pradeep Natarajan
| Challenge: | Large-scale, voice-based conversational assistants process each utterance through a multi-stage pipeline that includes wakeword detection, automatic speech recognition (ASR), natural language understanding (NLU), entity resolution, and textto-speech. |
| Approach: | They propose a method to identify customer implicitly satisfied with Alexa's responses by leveraging interpretations of model behavior. |
| Outcome: | The proposed approach produces statistically significant improvements in both offline and online tests. |