Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Track
Scalable and Robust Self-Learning for Skill Routing in Large-Scale Conversational AI Systems (2022.naacl-industry)
Copied to clipboard
| Challenge: | Existing methods to enable skill routing do not scale in terms of the number of skills and skill on-boarding. |
| Approach: | They propose a model-based approach to enable natural conversation by allowing frequent policy updates . they propose an annotation-based system, rule-based model, and bandit-based learning . |
| Outcome: | The proposed method is scalable and cost-effective, the authors show . they show that it can improve the user experience without abrupt policy changes . |
CREATER: CTR-driven Advertising Text Generation with Controlled Pre-Training and Contrastive Fine-Tuning (2022.naacl-industry)
Copied to clipboard
| Challenge: | a paper focuses on automatically generating the text of an ad to capture user interest for achieving higher click-through rate. |
| Approach: | They propose a CTR-driven advertising text generation approach to generate ad texts based on user reviews. |
| Outcome: | The proposed approach outperforms existing approaches on industrial datasets and on large-scale unpaired reviews. |
Augmenting Poetry Composition with Verse by Verse (2022.naacl-industry)
Copied to clipboard
| Challenge: | a new approach to poetry generation has been developed that allows an AI to generate a full poem by itself, thus writing in a closed system. |
| Approach: | They describe an AI poet that offers suggestions while a user is composing a poem . they use a generative model and a dual encoder model to offer the suggestions . |
| Outcome: | The proposed system can offer suggestions generated lines of verse while a user is composing a poem. |
AB/BA analysis: A framework for estimating keyword spotting recall improvement while maintaining audio privacy (2022.naacl-industry)
Copied to clipboard
| Challenge: | Keyword spotting systems that detect keywords in speech are difficult to evaluate under privacy constraints. |
| Approach: | They propose to use offline decoding to evaluate a candidate KWS model against a baseline model without requiring negative examples. |
| Outcome: | The proposed method improves the time, privacy, and cost of the evaluation and compares with real data. |
Temporal Generalization for Spoken Language Understanding (2022.naacl-industry)
Copied to clipboard
| Challenge: | Spoken Language Understanding models are usually trained offline on historical data, but must perform well on incoming user requests after deployment. |
| Approach: | They propose different strategies for achieving good temporal generalization . they focus on temporal drift, where the distribution of utterances may change . |
| Outcome: | The proposed model can perform well on unseen domains, e.g., upcoming data. |
An End-to-End Dialogue Summarization System for Sales Calls (2022.naacl-industry)
Copied to clipboard
| Challenge: | Summarizing sales calls is a routine task performed manually by salespeople. |
| Approach: | They propose a production system which combines generative models fine-tuned for customer-agent setting, with a human-in-the-loop user experience for an interactive summary curation process. |
| Outcome: | The proposed system can handle training data scarcity and privacy constraints in an industrial setting. |
Controlled Data Generation via Insertion Operations for NLU (2022.naacl-industry)
Copied to clipboard
| Challenge: | a new approach to annotate live traffic is emerging to be cost-effective and efficient . manual data annotation is expensive and not preferred for meeting customer privacy expectations . |
| Approach: | They propose a targeted synthetic data generation technique by inserting tokens into a given semantic signature. |
| Outcome: | The proposed approach achieves the same accuracy as training with all available data on a voice assistant dataset. |
Easy and Efficient Transformer: Scalable Inference Solution For Large NLP Model (2022.naacl-industry)
Copied to clipboard
Gongzheng Li, Yadong Xi, Jingzhen Ding, Duan Wang, Ziyang Luo, Rongsheng Zhang, Bai Liu, Changjie Fan, Xiaoxi Mao, Zeng Zhao
| Challenge: | Recent studies show that transformer-based models are effective over many tasks, but they are expensive to deploy in the industrial application. |
| Approach: | They propose a transformer-based inference solution that optimizes kernels for long inputs and large hidden sizes and a flexible CUDA memory manager to reduce the memory footprint when deploying a large model. |
| Outcome: | The proposed solution achieves an average speedup of 1.40-4.20x on the transformer decoder layer with an A100 GPU. |
Aspect-based Analysis of Advertising Appeals for Search Engine Advertising (2022.naacl-industry)
Copied to clipboard
| Challenge: | ad creators must consider various aspects of advertising appeals such as price, product features, and quality in their ac work. |
| Approach: | They propose to use a dataset of ad texts to explore the effective aspects of advertising appeals (A3) for different industries to assist a search engine ap creators. |
| Outcome: | The proposed model can detect aspects of ad texts and help them estimate their performance. |
Self-supervised Product Title Rewrite for Product Listing Ads (2022.naacl-industry)
Copied to clipboard
| Challenge: | Existing work has investigated the title optimization for Product Listing Ads (PLAs) however, little work has examined the effectiveness of this method. |
| Approach: | They propose a method to rewrite product listing ads titles without considering the fluency and information priority. |
| Outcome: | The proposed solution reduces the cost and improves CTR in the offline test and real-world online test by a large amount. |
Efficient Semi-supervised Consistency Training for Natural Language Understanding (2022.naacl-industry)
Copied to clipboard
| Challenge: | Manually labeled training data is expensive, noisy, and often scarce . semi-supervised learning methods can be used to improve model performance . |
| Approach: | They explore different methods for consistency training on unlabeled data . they use human paraphrasing, back-translation, and dropout to augment unlabed data. |
| Outcome: | The proposed methods outperform purely supervised learning on unlabeled data. |
Distantly Supervised Aspect Clustering And Naming For E-Commerce Reviews (2022.naacl-industry)
Copied to clipboard
| Challenge: | Product aspect extraction from reviews is a critical task for e-commerce services . scale of reviews makes human review at ecommerce scale infeasible. |
| Approach: | They propose automated methods for extracting aspect phrases from reviews . they train transformer based sentence embeddings that are aware of unique e-commerce language characteristics . |
| Outcome: | The proposed method improves the Silhouette Score by 64% compared to the state-of-the-art model . human review at e-commerce scale is infeasible due to the scale of the reviews . |
Local-to-global learning for iterative training of production SLU models on new features (2022.naacl-industry)
Copied to clipboard
| Challenge: | In many real-world NLP systems, new data becomes available with time and there is a need to refresh the model. |
| Approach: | They propose to adapt a local-to-global learning schedule to production settings where full data is not available at initial training iterations. |
| Outcome: | The proposed model improves model error rates by 7.3% and saves up to 25% training time for individual iterations. |
CULG: Commercial Universal Language Generation (2022.naacl-industry)
Copied to clipboard
| Challenge: | Pre-trained language models have improved performance for many NLP tasks in finance and healthcare. |
| Approach: | They propose a large-scale commercial universal language generation model which is pre-trained on a corpus drawn from 10 markets across 7 languages. |
| Outcome: | The proposed model outperforms other models on commercial generation tasks and on other markets, languages, and tasks. |
Constraining word alignments with posterior regularization for label transfer (2022.naacl-industry)
Copied to clipboard
| Challenge: | Unsupervised word alignments are not always possible in industrial NLP pipelines, where multilingual annotation guidelines are complex and deviate from semantic consistency due to various factors. |
| Approach: | They propose to constrain word alignment models to remain consistent with both source and target annotation guidelines by leveraging posterior regularization and labeled examples. |
| Outcome: | The proposed model improves on the multiATIS++ dataset over AWESoME, and even a small amount of target language annotations can help. |
Explaining the Effectiveness of Multi-Task Learning for Efficient Knowledge Extraction from Spine MRI Reports (2022.naacl-industry)
Copied to clipboard
Arijit Sehanobish, McCullen Sandora, Nabila Abraham, Jayashri Pawar, Danielle Torres, Anasuya Das, Murray Becker, Richard Herzog, Benjamin Odry, Ron Vianu
| Challenge: | Pretrained Transformer based models finetuned on domain specific corpora have changed the landscape of NLP but training or fine-tuning these models for individual tasks can be time consuming and resource intensive. |
| Approach: | They propose to use pretrained Transformer based models finetuned on domain specific corpora to train models for individual tasks. |
| Outcome: | The proposed model can match the performance of a task specific model when the task specific models show similar representations across all of their hidden layers and their gradients are aligned, i.e. their gradient follows the same direction. |
FPI: Failure Point Isolation in Large-scale Conversational Assistants (2022.naacl-industry)
Copied to clipboard
| Challenge: | Large-scale conversational assistants can cause errors in their modules . a machine learning system can analyze large volumes of data and isolate the source of error . |
| Approach: | They propose a machine learning system that embeds incoming request and context using pre-trained transformer models and encodes additional metadata features to output failure point predictions. |
| Outcome: | The proposed system obtains 92.2% of human performance while scaling to analyze the entire traffic in 8 different languages of a large-scale conversational assistant. |
Asynchronous Convergence in Multi-Task Learning via Knowledge Distillation from Converged Tasks (2022.naacl-industry)
Copied to clipboard
Weiyi Lu, Sunny Rajagopalan, Priyanka Nigam, Jaspreet Singh, Xiaodi Sun, Yi Xu, Belinda Zeng, Trishul Chilimbi
| Challenge: | Multi-task learning (MTL) aims to solve multiple tasks by sharing a base representation among them. |
| Approach: | They propose an approach that allows for "asynchronous" convergence among the tasks where each task can converge on its own schedule. |
| Outcome: | The proposed method outperforms existing methods in two 5-task MTL setups. |
Augmenting Training Data for Massive Semantic Matching Models in Low-Traffic E-commerce Stores (2022.naacl-industry)
Copied to clipboard
Ashutosh Joshi, Shankar Vishwanath, Choon Teo, Vaclav Petricek, Vishy Vishwanathan, Rahul Bhagat, Jonathan May
| Challenge: | Existing methods to augment training data for e-commerce stores using behavioral data are limited in low-traffic stores . eXtreme multi-label classification systems require large amounts of customer behavior data . |
| Approach: | They propose a technique that augments behavioral training data via query reformulation . they use an example semantic matching model from the e-commerce store AL-XMC . |
| Outcome: | The proposed method improves quality of the AL-XMC model over a baseline model. |
Retrieval Based Response Letter Generation For a Customer Care Setting (2022.naacl-industry)
Copied to clipboard
| Challenge: | Letter-like communications are a major means of customer relationship management . despite advances in natural language processing, the task of generating a response is time-consuming . |
| Approach: | They propose a deep-learning based response letter generation framework that uses data augmentation to retrieve knowledge from historical responses and utilize it to generate an appropriate response. |
| Outcome: | The proposed model outperforms baselines by significant margins while producing consistent and informative responses. |
Medical Coding with Biomedical Transformer Ensembles and Zero/Few-shot Learning (2022.naacl-industry)
Copied to clipboard
| Challenge: | Medical coding (MC) is an essential pre-requisite for reliable data retrieval and reporting. |
| Approach: | They propose a method to classify medical terms into standardized alphanumerical terms and codes . they use a combination of traditional BERT-based classification and a zero/few-shot learning approach . |
| Outcome: | The proposed approach outperforms baselines in the few-shot regime. |
Knowledge extraction from aeronautical messages (NOTAMs) with self-supervised language models for aircraft pilots (2022.naacl-industry)
Copied to clipboard
| Challenge: | During pre-flight briefings, aircraft pilots analyse a long list of NOTAMs . the messages are usually written in the English language, but the phrasing is very special . |
| Approach: | They pretrain language models derived from BERT on circa 1 million unlabeled NOTAMs . they reuse the learnt representations on three downstream tasks valuable for pilots - criticality prediction, named entity recognition and translation into a structured language called Airlang. |
| Outcome: | The proposed language model can be used on criticality prediction, named entity recognition and translation into a structured language called Airlang. |
Intent Discovery for Enterprise Virtual Assistants: Applications of Utterance Embedding and Clustering to Intent Mining (2022.naacl-industry)
Copied to clipboard
| Challenge: | Existing approaches to clustering unlabeled utterances are based on transformerbased sentence embedding methods. |
| Approach: | They propose a semantic embedding approach that can be leveraged to identify clusters of utterances that correspond to unhandled intents. |
| Outcome: | The proposed approach can identify clusters of utterances that correspond to unhandled intents from a large collection of enterprise virtual assistant data using a multi-task softmax loss. |
ReFinED: An Efficient Zero-shot-capable Approach to End-to-End Entity Linking (2022.naacl-industry)
Copied to clipboard
| Challenge: | Entity linking is the task of recognising mentions of entities in unstructured text documents and linking them to the corresponding entities in a Knowledge Base (KB) the largest public EL dataset is Wikipedia, which covers just 3% of the entities in Wikidata. |
| Approach: | They propose a model which performs mention detection, fine-grained entity typing, and entity disambiguation for all mentions within a document in a single forward pass. |
| Outcome: | The proposed model outperforms state-of-the-art methods on standard datasets by an average of 3.7 F1 and can generalise to large-scale knowledge bases such as Wikidata and zero-shot entity linking. |
Lightweight Transformers for Conversational AI (2022.naacl-industry)
Copied to clipboard
| Challenge: | Commercial dialogue systems typically require a small footprint and fast execution time, but recent trends are in the other direction, resulting in difficulties in model deployment. |
| Approach: | They build Transformer-based Language Models from scratch on large corpora of conversational data and compare their performance against BERT and other strong baselines on dialogue probing tasks. |
| Outcome: | The proposed model outperforms existing models on dialogue probing tasks and can be fine-tuned on a single consumer GPU card. |
NER-MQMRC: Formulating Named Entity Recognition as Multi Question Machine Reading Comprehension (2022.naacl-industry)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a task of locating and classifying entities mentioned in unstructured text into predefined categories. |
| Approach: | They propose to use a BERT-based multi-question MRC task where multiple questions (one question per entity) are considered at the same time for a single text. |
| Outcome: | The proposed architecture leads to 2.5 times faster training and 2.3 times faster inference on three NER datasets. |
What Do Users Care About? Detecting Actionable Insights from User Feedback (2022.naacl-industry)
Copied to clipboard
| Challenge: | a large amount of data can be used to extract actionable insights from user feedback . however, the data is unstructured and voluminous, and is underutilized for most users . |
| Approach: | They propose an unsupervised method for finding actionable insights from user feedback . they cluster data into groups containing coherent insights, followed by theme detection . |
| Outcome: | The proposed approach outperforms baselines on two real-world user feedback datasets and one academic dataset. |
CTM - A Model for Large-Scale Multi-View Tweet Topic Classification (2022.naacl-industry)
Copied to clipboard
| Challenge: | Existing methods to classify social media posts into topics have been used to class up documents into topics. |
| Approach: | They propose a neural model that automatically associates social media posts with topics to solve these challenges. |
| Outcome: | The proposed model outperforms existing methods in the context of Twitter where the topic space is 10 times larger with potentially multiple topic associations per Tweet. |
Developing a Production System for Purpose of Call Detection in Business Phone Conversations (2022.naacl-industry)
Copied to clipboard
| Challenge: | a commercial system detects Purpose of Call statements in call transcripts . the model is based on a set of rules and a neural model . |
| Approach: | They propose a system to detect Purpose of Call statements in English business call transcripts in real time. |
| Outcome: | The proposed model achieves 88.6 F1 on average in various types of business calls and has low inference time. |
Adversarial Text Normalization (2022.naacl-industry)
Copied to clipboard
| Challenge: | Text-based adversarial attacks are becoming more commonplace and accessible to general internet users. |
| Approach: | They propose a method that restores baseline performance on attacked content with low computational overhead. |
| Outcome: | The proposed method restores baseline performance on attacked content with low computational overhead. |
Constraint-based Multi-hop Question Answering with Knowledge Graph (2022.naacl-industry)
Copied to clipboard
| Challenge: | Recent work addresses multi-hop KGQA, which requires reasoning across numerous edges of the KG. |
| Approach: | They propose to use KG embeddings to reduce KG sparsity by performing missing link prediction. |
| Outcome: | Empirical results show that the proposed method produces state-of-the-art results on three KGQA datasets. |
Fast Bilingual Grapheme-To-Phoneme Conversion (2022.naacl-industry)
Copied to clipboard
| Challenge: | Autoregressive transformers (ART)-based grapheme-to-phoneme (G2P) models have been proposed for bi/multilingual text-to speech systems. |
| Approach: | They propose a bilingual grapheme-to-phoneme (G2P) model with autoregressive transformers for fast and exact decoding and data augmentation for predicting output length. |
| Outcome: | The proposed model achieves better performance than the previous model and 2700% faster inference speed. |
Knowledge Extraction From Texts Based on Wikidata (2022.naacl-industry)
Copied to clipboard
| Challenge: | Existing knowledge extraction pipelines for English are not suitable for enterprise use. |
| Approach: | They propose a knowledge extraction pipeline for English which can be further used for building an entreprise-specific knowledge base. |
| Outcome: | The proposed pipeline can be used to build an entreprise-specific knowledge base. |
AIT-QA: Question Answering Dataset over Complex Tables in the Airline Industry (2022.naacl-industry)
Copied to clipboard
Yannis Katsis, Saneem Chemmengath, Vishwajeet Kumar, Samarth Bharadwaj, Mustafa Canim, Michael Glass, Alfio Gliozzo, Feifei Pan, Jaydeep Sen, Karthik Sankaranarayanan, Soumen Chakrabarti
| Challenge: | Table Question Answering (Table QA) systems have been shown to be highly accurate when trained and tested on open-domain datasets built on top of Wikipedia tables. |
| Approach: | They propose a domain-specific Table QA test dataset to test Table Question Answering systems on open-domain datasets built on top of Wikipedia tables. |
| Outcome: | The proposed methods are highly accurate when tested on open-domain datasets built on top of Wikipedia tables. |
Parameter-efficient Continual Learning Framework in Industrial Real-time Text Classification System (2022.naacl-industry)
Copied to clipboard
Tao Zhu, Zhe Zhao, Weijie Liu, Jiachi Liu, Yiren Chen, Weiquan Mao, Haoyan Liu, Kunbo Ding, Yudong Li, Xuefeng Yang
| Challenge: | Existing continual learning methods use data replay, parameter isolation and regularization to mitigate catastrophic forgetting. |
| Approach: | They propose a parameter-efficient continual learning framework that updates parameters offline and then trains using an online regularization method. |
| Outcome: | The proposed framework reduces catastrophic forgetting and saves the model with the changed parameters instead of all parameters. |
Self-Aware Feedback-Based Self-Learning in Large-Scale Conversational AI (2022.naacl-industry)
Copied to clipboard
| Challenge: | Large-scale conversational AI systems require constant update to adapt to changing customer behavior and trends . lack of self-awareness in feedback-based systems can cause degradation of performance . et al., e. alderman and scott k. d. argues that such systems are not scalable enough to sustain the rapid update pace of conversational systems. |
| Approach: | They propose a superposition-based model that reactively learns local-adaptive decision boundaries . they propose rewritings with a bi-variate beta setting to improve the model's performance . |
| Outcome: | The proposed model improves the PR-AUC by 27.45% and reduces relative defect reductions by 31.22% . the proposed model can adapt faster to changes in global preferences across a large number of customers . |
Fast and Light-Weight Answer Text Retrieval in Dialogue Systems (2022.naacl-industry)
Copied to clipboard
| Challenge: | a recent study shows that text retrieval can be used to find information relevant to user requests. |
| Approach: | They propose to use a corpus of text to search for relevant responses to user requests . they compare this approach to other methods that use intent detection . |
| Outcome: | a new approach can be used to search through a corpus of text to find relevant responses to user requests. |
BLINK with Elasticsearch for Efficient Entity Linking in Business Conversations (2022.naacl-industry)
Copied to clipboard
Md Tahmid Rahman Laskar, Cheng Chen, Aliaksandr Martsinovich, Jonathan Johnston, Xue-Yong Fu, Shashi Bhushan Tn, Simon Corston-Oliver
| Challenge: | Existing systems that align textual mentions of entities to knowledge bases are difficult to deploy in production environments. |
| Approach: | They propose a neural entity linking system that connects entities in business phone conversations to their corresponding Wikipedia and Wikidata entries. |
| Outcome: | The proposed system improves inference speed and memory consumption while maintaining high accuracy. |
Q2R: A Query-to-Resolution System for Natural-Language Queries (2022.naacl-industry)
Copied to clipboard
| Challenge: | Existing text ranking methods are expensive since they require a parametric classifier to retrieve a small D D. |
| Approach: | They propose a system that combines direct classification with standard content-based retrieval approaches to significantly improve the relevance of retrieved documents. |
| Outcome: | The proposed system improves the relevance of retrieved documents by using a novel Q2R orchestration framework. |
Identifying Corporate Credit Risk Sentiments from Financial News (2022.naacl-industry)
Copied to clipboard
| Challenge: | Existing methods to assess default probabilities are tedious and time-consuming due to the deluge of news coverage for financial institutions. |
| Approach: | They propose a deep learning-powered approach to automate news analysis and credit adverse events detection to score the credit sentiment associated with a company. |
| Outcome: | The proposed system leverages news extraction and data enrichment with targeted sentiment entity recognition to detect companies and text classification to identify credit events. |