Predicting Customer Satisfaction with Soft Labels for Ordinal Classification (2023.acl-industry)
Copied to clipboard
| Challenge: | a typical call center only responds to 8% of customers with a customer satisfaction survey . a predictive algorithm that infers CSAT on the 1-5 scale is needed to minimize this data sparsity and response bias. |
| Approach: | They propose an algorithm that infers CSAT on 1-5 scale on inbound calls to the call center . they reframe the problem into a binary class and map it back to five classes . |
| Outcome: | The proposed model is able to support keycustomer workflows with high accuracy overmillions of calls a month. |
Similar Papers
Mitigating Shortcuts in Language Models with Soft Label Encoding (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent studies have shown that large language models rely on spurious correlations in the data for natural language understanding (NLU) tasks. |
| Approach: | They propose a framework for debiasing shortcuts and a dummy class to encode shortcuts into a model and use it to generate soft labels. |
| Outcome: | The proposed framework significantly improves out-of-distribution generalization while maintaining satisfactory in-district accuracy. |
Enhancing Automatic Readability Assessment with Pre-training and Soft Labels for Ordinal Regression (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models do not exploit ordinal nature of difficulty grades and make little effort for initialization to facilitate fine-tuning. |
| Approach: | They propose a readability assessment task that assigns a difficulty grade to a text . they use ordinal regression and pairwise relative text difficulty to train the model . |
| Outcome: | The proposed model outperforms competitive neural models and statistical classifiers on most datasets. |
Weakly supervised hierarchical multi-task classification of customer questions (2023.acl-industry)
Copied to clipboard
Jitenkumar Rana, Promod Yenigalla, Chetan Aggarwal, Sandeep Sricharan Mukku, Manan Soni, Rashmi Patange
| Challenge: | Identifying granular and actionable topics from customer questions helps improve the overall customer experience. |
| Approach: | They propose a weakly supervised Hierarchical Multi-task Classification Framework to identify granular topics from customer questions . a clustering based taxonomy creation and data labeling module is used to create taxonomies and labelled data with minimal supervision. |
| Outcome: | The proposed model achieves 13% better accuracy over single-task classification frameworks . it can adapt to constantly evolving taxonomy without need of re-training . |
Learning to Correct Noisy Labels for Fine-Grained Entity Typing via Co-Prediction Prompt Tuning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show that noise correction in fine-grained entity typing improves quality of training samples. |
| Approach: | They propose a method that leverages multiple prediction results to correct noisy labels . they integrate prediction results and utilize a differentiated margin to identify inaccurate labels a . |
| Outcome: | The proposed model improves quality of training samples annotated using distant supervision, ChatGPT, and crowdsourcing. |
Beyond Black & White: Leveraging Annotator Disagreement via Soft-Label Multi-Task Learning (2021.naacl-main)
Copied to clipboard
| Challenge: | Prior work shows that disagreement between annotators can be useful in training models. |
| Approach: | They propose to use disagreements as an auxiliary task in a multi-task neural network to incorporate disagreements into models. |
| Outcome: | The proposed method significantly improves performance on NLP tasks beyond the standard approach and prior work. |
From TextBlob to LLM Agents: Sentiment Model Selection for B2B Technical Support with CSAT Ground Truth (2026.acl-industry)
Copied to clipboard
| Challenge: | a dedicated single-task LLM agent reduces neutral bias from 69% to 22%, improving MCC from -0.018 to 0.347 . only 4.88% of tickets receive negative satisfaction ratings . |
| Approach: | They evaluate sentiment models for customer satisfaction prediction in B2B technical support . they use a complete population of CSAT-rated tickets from 100+ organizations . |
| Outcome: | The proposed model performs better than the most expensive model, with a lower neutrality and lower recall than the budget model. |
Beyond Averages: Learning with Annotator Disagreement in STS (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to capture and model disagreement in Semantic Textual Similarity (STS) ignore label dispersion and incentivize models to ignore uncertainty crucial for practical settings. |
| Approach: | They propose to capture and model disagreement in Semantic Textual Similarity (STS) a lightweight truncated Gaussian head and a cross-encoder are used to model disagreement . |
| Outcome: | The proposed approach improves accuracy and calibration of models to human judgments. |
Robust Machine Reading Comprehension by Learning Soft labels (2020.coling-main)
Copied to clipboard
| Challenge: | Neural models have achieved great success on the task of machine reading comprehension, which are typically trained on hard labels. |
| Approach: | They propose a robust training method for machine reading comprehension models to address label sparseness problem by using three strategies to train models on soft labels. |
| Outcome: | The proposed method improves the baseline model performance and achieves state-of-the-art performance on NewsQA and QUOREF. |
A Deep Reinforced Sequence-to-Set Model for Multi-Label Classification (P19-1)
Copied to clipboard
| Challenge: | Multi-label classification (MLC) aims to assign multiple labels to each sample. |
| Approach: | They propose a sequence-to-set model that is trained via reinforcement learning and rewards feedback independent of the label order. |
| Outcome: | The proposed model outperforms baseline models and reduces sensitivity to label order. |
Systematic Evaluation of Predictive Fairness (2022.aacl-main)
Copied to clipboard
| Challenge: | Several methods have been proposed to mitigate bias in training on biased datasets. |
| Approach: | They propose to examine the effect of target class imbalance and stereotyping on model performance by analyzing binary classification, profession prediction and regression tasks. |
| Outcome: | The proposed methods show that data conditions have a strong influence on relative model performance. |