Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: Industry Track

A Hybrid Approach to Cross-lingual Product Review Summarization (2022.emnlp-industry)

Challenge:	Recent research shows that large language models pretrained using unsupervised approaches can achieve significant performance improvement on many downstream tasks.
Approach:	They propose an unsupervised approach to fine-tuning large language models using unsupervised approaches to many downstream tasks.
Outcome:	The proposed approach improves on four e-commerce applications and can achieve an average improvement of 10% in few-shot settings and 3.7% in data-rich settings over the standard approach.

Copied to clipboard

Saleh Soltan, Victor Soto, Ke Tran, Wael Hamza

Text Generation

Challenge:	Existing methods for summarizing product reviews with thousands of reviews are inefficient and time consuming.
Approach:	They propose an unsupervised extractive step and a supervised abstractive step to generate a short summary in any language.
Outcome:	The proposed model is as good as human written summaries in coherence, informativeness, non-redundancy, and fluency as human summary summators.

Augmenting Operations Research with Auto-Formulation of Optimization Models From Problem Descriptions (2022.emnlp-industry)

Copied to clipboard

Rindra Ramamonjison, Haley Li, Timothy Yu, Shiqi He, Vishnu Rengan, Amin Banitalebi-dehkordi, Zirui Zhou, Yong Zhang

Challenge:	Existing systems for operations research use NLP to suggest formulations of optimization problems.
Approach:	They propose an augmented intelligence system that can be used to simplify and enhance the modeling experience for operations research.
Outcome:	The proposed system validates and edits the proposed formulations with a dataset of linear programming problems drawn from various application domains.

Knowledge Distillation based Contextual Relevance Matching for E-commerce Product Search (2022.emnlp-industry)

Copied to clipboard

Ziyang Liu, Chaokun Wang, Hao Feng, Lingfei Wu, Liqun Yang

Accelerating the Discovery of Semantic Associations from Medical Literature: Mining Relations Between Diseases and Symptoms (2022.emnlp-industry)

Challenge:	Existing approaches to e-commerce relevance matching ignore bipartite graphs in logs . experimental results show that proposed method improves human relevance judgment .
Approach:	They propose an efficient knowledge distillation framework for e-commerce relevance matching to exploit the advantages of Transformer-style and classical relevance matching models.
Outcome:	The proposed method significantly improves human relevance judgment on large-scale real-world data.

Copied to clipboard

Alberto Purpura, Francesca Bonin, Joao Bettencourt-silva

PENTATRON: PErsonalized coNText-Aware Transformer for Retrieval-based cOnversational uNderstanding (2022.emnlp-industry)

Challenge:	Existing methods to extract semantic associations from medical literature do not take into account the semantics of sentences from which entity co-occurrences are extracted.
Approach:	They propose a system for the automatic discovery of semantic associations between different entities such as diseases and their symptoms using a semantic network and a binary relation classification model trained with distant supervision.
Outcome:	The proposed system validates the extracted associations against a publicly available list of disease-symptom pairs against 14M PubMed abstracts.

Copied to clipboard

Niranjan Uma Naresh, Ziyan Jiang, Ankit Ankit, Sungjin Lee, Jie Hao, Xing Fan, Chenlei Guo

Machine translation impact in E-commerce multilingual search (2022.emnlp-industry)

Challenge:	In a large fraction of the global traffic from smart digital assistants, frictions in dialogues may be attributed to incorrect understanding of the entities in a user's query due to factors including ambiguous mentions, mispronunciation, background noise and faulty on-device signal processing.
Approach:	They propose a parametric transformer-based language model to learn patterns from in-session customer-device interactions coupled with a non-parametric personalized entity index to compute the correct query.
Outcome:	The proposed system improves on the existing system and shows that it can learn the correct query from in-session customer-device interactions.

Copied to clipboard

Bryan Zhang, Amita Misra

Multilinguality

Challenge:	Existing studies have demonstrated that cross-lingual information retrieval performance is highly dependent on query translation quality.
Approach:	They investigate whether improving query translation quality yields little or no benefit to further improve retrieval performance.
Outcome:	The proposed methods compare query translations for multiple language pairs and identify the most promising language pairs to invest and improve.

Ask-and-Verify: Span Candidate Generation and Verification for Attribute Value Extraction (2022.emnlp-industry)

Copied to clipboard

Yifan Ding, Yan Liang, Nasser Zalmout, Xian Li, Christan Grant, Tim Weninger

Consultation Checklists: Standardising the Human Evaluation of Medical Note Generation (2022.emnlp-industry)

Challenge:	Existing reading comprehension models can over-generate attribute values which hinders precision.
Approach:	They propose a product attribute value extraction task that captures key factual information from product descriptions and a new end-to-end pipeline framework called Ask-and-Verify.
Outcome:	The proposed framework outperforms existing models by up to 3.1% F1 absolute improvement points while scaling to thousands of attributes.

Copied to clipboard

Aleksandar Savkov, Francesco Moramarco, Alex Papadopoulos Korfiatis, Mark Perera, Anya Belz, Ehud Reiter

Challenge:	Existing studies have shown that note generation is difficult due to subjective nature of many aspects of output quality.
Approach:	They propose a protocol that aims to increase objectivity by grounding evaluations in Consultation Checklists, which are created in a preliminary step and then used as a common point of reference during quality assessment.
Outcome:	The proposed protocol shows that the evaluations produced in the study are more objective than the original human note.

Towards Need-Based Spoken Language Understanding Model Updates: What Have We Learned? (2022.emnlp-industry)

Copied to clipboard

Quynh Do, Judith Gaspers, Daniil Sorokin, Patrick Lehnen

Challenge:	In productionized machine learning systems, online model performance deteriorates when there is a distributional drift between offline training and online data.
Approach:	They propose a need-based retraining strategy guided by an efficient drift detector . they propose overlapping model releases, observation limitation and lack of annotated resources at runtime .
Outcome:	The proposed strategy reduces the cost of retraining models at fixed intervals . the proposed strategy can detect drifts when the model is applied on a new data set .

Knowledge Distillation Transfer Sets and their Impact on Downstream NLU Tasks (2022.emnlp-industry)

Copied to clipboard

Charith Peris, Lizhen Tan, Thomas Gueudre, Turan Gojayev, Pan Wei, Gokmen Oz

Exploiting In-Domain Bilingual Corpora for Zero-Shot Transfer Learning in NLU of Intra-Sentential Code-Switching Chatbot Interactions (2022.emnlp-industry)

Challenge:	Domain Classification (DC) and Intent Classification/Named Entity Recognition (ICNER) are the most common methods for reducing teacher-student knowledge into manageable sizes for low-latency downstream applications.
Approach:	They investigate whether distillation from a generic LM benefits downstream tasks . a domain classification and a task-specific data set are used to fine tune the model .
Outcome:	The proposed model improves across tasks and test sets when only task-specific data is used.

Copied to clipboard

Maia Aguirre, Manex Serras, Laura García-sardiña, Jacobo López-fernández, Ariane Méndez, Arantza Del Pozo

Multilinguality

Challenge:	Multilingual speakers outnumber monolingual speakers in the world . CS is a frequent habit in both spoken and written informal communications .
Approach:	They evaluate the efficacy of cross-lingual transfer learning with mBERT for NLU on a Basque-Spanish CS chatbot corpus.
Outcome:	The proposed model outperforms models trained on Basque and Spanish without CS on a basque-Spanish chatbot corpus.

Calibrating Imbalanced Classifiers with Focal Loss: An Empirical Study (2022.emnlp-industry)

Copied to clipboard

Cheng Wang, Jorge Balazs, György Szarvas, Patrick Ernst, Lahari Poddar, Pavel Danchenko

Unsupervised training data re-weighting for natural language understanding with local distribution approximation (2022.emnlp-industry)

Challenge:	Imbalanced data distributions can cause models to overfit to majority classes and output unreliable (mostly overconfident) predictions.
Approach:	They propose to streamline the model development and deployment using focal loss to address imbalanced data distributions.
Outcome:	The proposed model training with focal loss improves calibration and accuracy compared to standard cross-entropy loss.

Copied to clipboard

Jose Garrido Ramas, Dieu-thu Le, Bei Chen, Manoj Kumar, Kay Rottmann

Cross-Encoder Data Annotation for Bi-Encoder Based Product Matching (2022.emnlp-industry)

Challenge:	a distribution mismatch between offline training and live data can cause biases . cyclic seasonality shifts, and changing pool of users can contribute to this problem .
Approach:	They propose an unsupervised approach to mitigate offline training data sampling bias . they propose a local distribution approximation in the pre-trained embedding space .
Outcome:	The proposed approach mitigates the offline training data sampling bias in multiple NLU tasks without additional annotation.

Copied to clipboard

Justin Chiu, Keiji Shinzato

Deploying a Retrieval based Response Model for Task Oriented Dialogues (2022.emnlp-industry)

Challenge:	Existing approaches to match seller listed items to appropriate product are computationally heavy and require computational resources.
Approach:	They propose a technique to annotate or refine human annotated training data for bi-encoder models using a cross-encoding model.
Outcome:	The proposed approach improves 4% absolute accuracy when no training data is available and 2% when annotated training data exists.

Copied to clipboard

Lahari Poddar, György Szarvas, Cheng Wang, Jorge Balazs, Pavel Danchenko, Patrick Ernst

Tackling Temporal Questions in Natural Language Interface to Databases (2022.emnlp-industry)

Challenge:	a task-oriented dialogue system needs high conversational capability and can be easily adaptable to changing situations.
Approach:	They propose a retrieval-based conversational model that can rank a large set of responses . they propose supervised training and fine-tuning on limited data collected through a human-in-the-loop platform .
Outcome:	The proposed model can scale to rank a large set of responses in real-world situations.

Copied to clipboard

Ngoc Phuoc An Vo, Octavian Popescu, Irene Manotas, Vadim Sheinin

Multi-Tenant Optimization For Few-Shot Task-Oriented FAQ Retrieval (2022.emnlp-industry)

Challenge:	Temporal aspect is one of the most challenging areas in Natural Language Interface to Databases (NLIDB).
Approach:	They propose a dataset with accompanied databases supporting temporal questions in NLIDB.
Outcome:	The proposed dataset helps two models learn and improve in temporal aspect.

Copied to clipboard

Asha Vishwanathan, Rajeev Warrier, Gautham Vadakkekara Suresh, Chandra Shekhar Kandpal

Low-Resource NLP

Challenge:	Business-specific Frequently Asked Questions (FAQs) are important part of many taskoriented dialog systems today.
Approach:	They propose a method to scale multi-tenant FAQ applications in real-world context . they use query-Question similarity and few-shot intent detection techniques .
Outcome:	The proposed approach scales multi-tenant FAQ applications in real-world context . it uses fine-tuning of the last layer in sentence Bi-Encoders along with tenant-specific weight switching.

Iterative Stratified Testing and Measurement for Automated Model Updates (2022.emnlp-industry)

Copied to clipboard

Elizabeth Dekeyser, Nicholas Comment, Shermin Pei, Rajat Kumar, Shruti Rai, Fengtao Wu, Lisa Haverty, Kanna Shimizu

Challenge:	Automating updates to machine learning systems is an important but understudied challenge in AutoML.
Approach:	They propose a framework that relies on iterative model building coupled with data-shape stratified model testing and improvement to improve model accuracy.
Outcome:	The proposed framework shows a 26% improvement in accuracy for new model use cases on a large-scale NLU system compared to a naive baseline and current cutting-edge methods.

SLATE: A Sequence Labeling Approach for Task Extraction from Free-form Inked Content (2022.emnlp-industry)

Copied to clipboard

Apurva Gandhi, Ryan Serrao, Biyi Fang, Gilbert Antonius, Jenna Hong, Tra My Nguyen, Sheng Yi, Ehi Nosakhare, Irene Shaffer, Soundararajan Srinivasan

Gaining Insights into Unrecognized User Utterances in Task-Oriented Dialog Systems (2022.emnlp-industry)

Challenge:	SLATE is a sequence labeling approach for extracting tasks from free-form content . past approaches for task extraction from typed content focus on building separate sentence-level task classification models.
Approach:	They propose a sequence labeling approach for extracting tasks from free-form content . they use a single, low-latency sequence labelling approach to perform sentence segmentation and classification .
Outcome:	The proposed model outperforms a baseline model and achieves 84.4% task F1 score and 88.4% boundary similarity score.

Copied to clipboard

Ella Rabinovich, Matan Vetzler, David Boaz, Vineet Kumar, Gaurav Pandey, Ateret Anaby Tavor

Dialogue Systems & Conversational Agents

Challenge:	Goal-oriented dialog systems fail to recognize the intent of natural language requests due to system errors, incomplete service coverage, or insufficient training.
Approach:	They propose an end-to-end pipeline for processing unrecognized user utterances, deployed in a commercial task-oriented dialog system, including a specifically-tailored clustering algorithm, a novel approach to cluster representative extraction, and cluster naming.
Outcome:	The proposed components show that they improve the performance of the proposed system in the analysis of unrecognized user requests.

CoCoID: Learning Contrastive Representations and Compact Clusters for Semi-Supervised Intent Discovery (2022.emnlp-industry)

Copied to clipboard

Qian Cao, Deyi Xiong, Qinlong Wang, Xia Peng

Tractable & Coherent Multi-Document Summarization: Discrete Optimization of Multiple Neural Modeling Streams via Integer Linear Programming (2022.emnlp-industry)

Challenge:	Existing approaches to intent discovery cluster novel intents with prior knowledge from intent-labeled data in a semi-supervised way.
Approach:	They propose a semi-supervised intent discovery framework CoCoID with two components . they propose to discriminate user utterance representation learning and intra-cluster knowledge distillation .
Outcome:	The proposed framework outperforms state-of-the-art intent discovery models by over 1.4 ACC and ARI points and 1.1 NMI points across four datasets.

Copied to clipboard

Litton J Kurisinkel, Nancy Chen

Grafting Pre-trained Models for Multimodal Headline Generation (2022.emnlp-industry)

Challenge:	Multi-document summarization generates summary of corpus of documents consisting of related topics.
Approach:	They propose a generic framework to jointly consider coherence and informativeness in multi-document summarization and offers provisions to replace individual components based on the domain of source text.
Outcome:	The proposed framework consistently performs better than baselines for objective metrics and human evaluation.

Copied to clipboard

Lingfeng Qiao, Chen Wu, Ye Liu, Haoyuan Peng, Di Yin, Bo Ren

Semi-supervised Adversarial Text Generation based on Seq2Seq models (2022.emnlp-industry)

Challenge:	Existing approaches to generate video headlines with pre-trained language models are labor intensive and impractical.
Approach:	They propose to graft the encoder from the pre-trained video-language model on the generative pre-trainer model and propose a consensus fusion mechanism for the integration of different components.
Outcome:	The proposed model achieves strong results on a brand-new dataset collected from real-world applications.

Copied to clipboard

Hieu Le, Dieu-thu Le, Verena Weber, Chris Church, Kay Rottmann, Melanie Bradford, Peter Chin

Is it out yet? Automatic Future Product Releases Extraction from Web Data (2022.emnlp-industry)

Challenge:	In contrast, adversarial training has been used in computer vision to improve models’ robustness due to the discrete nature of text.
Approach:	They propose a way to generate adversarial samples by using pseudo-labeled in-domain text data to train a seq2seq model for adversarials and combine it with paraphrase detection.
Outcome:	The proposed model generates realistic and relevant adversarial samples compared to other state-of-the-art models and recovers up to 70% of errors.

Copied to clipboard

Gilad Fuchs, Ido Ben-shaul, Matan Mandelbrod

Automatic Scene-based Topic Channel Construction System for E-Commerce (2022.emnlp-industry)

Challenge:	Identifying product releases in advance is valuable for E-Commerce marketplaces and retailers.
Approach:	They propose a ML-powered pipeline to automatically identify future product releases from web data.
Outcome:	The proposed pipeline can identify future product releases and rank their predicted demand from unstructured web pages.

Copied to clipboard

Peng Lin, Yanyan Zou, Lingfei Wu, Mian Ma, Zhuoye Ding, Bo Long

SpeechNet: Weakly Supervised, End-to-End Speech Recognition at Industrial Scale (2022.emnlp-industry)

Challenge:	Recent scene marketing has proved effective for offline shopping.
Approach:	They propose a novel product form, scene-based topic channel, which consists of a list of diverse products belonging to the same usage scenario and a topic title that describes the scenario with marketing words.
Outcome:	The proposed system can be automated and tested on a real-world e-commerce recommendation platform.

Copied to clipboard

Raphael Tang, Karun Kumar, Gefei Yang, Akshat Pandey, Yajie Mao, Vladislav Belyaev, Madhuri Emmadi, Craig Murray, Ferhan Ture, Jimmy Lin

Controlled Language Generation for Language Learning Items (2022.emnlp-industry)

Challenge:	End-to-end automatic speech recognition systems require thousands of hours of manual annotation and heavyweight computation to perform inference.
Approach:	They propose to use a third-party ASR system as a weak supervision source and labeling functions derived from implicit user feedback to reduce human labor.
Outcome:	The proposed system improves word-error rate and speed up 600% over third-party ASR.

Copied to clipboard

Kevin Stowe, Debanjan Ghosh, Mengxuan Zhao

Text Generation

Challenge:	Recent advances in pre-trained language models have resulted in success in generating fluent English text.
Approach:	They propose to employ natural language generation to rapidly generate English language items . they experiment with deep pretrained models and develop methods for controlling items for factors relevant in language learning .
Outcome:	The proposed framework shows high grammatically scores for all models and higher complexity over baseline models.

Improving Text-to-SQL Semantic Parsing with Fine-grained Query Understanding (2022.emnlp-industry)

Copied to clipboard

Jun Wang, Patrick Ng, Alexander Hanbo Li, Jiarong Jiang, Zhiguo Wang, Bing Xiang, Ramesh Nallapati, Sudipta Sengupta

Programming Languages in NLP

Challenge:	Recent research on Text-to-SQL semantic parsing relies on parser or heuristic based approach to understand natural language query.
Approach:	They propose a general-purpose, modular neural semantic parsing framework that is based on token-level fine-grained query understanding.
Outcome:	The proposed framework outperforms the state-of-the-art model by 2.7% on a WikiTableQuestions test set.

Unsupervised Dense Retrieval for Scientific Articles (2022.emnlp-industry)

Copied to clipboard

Dan Li, Vikrant Yadav, Zubair Afzal, George Tsatsaronis

Learning Geolocations for Cold-Start and Hard-to-Resolve Addresses via Deep Metric Learning (2022.emnlp-industry)

Challenge:	Existing lexical search models suffer from lexica gap problems and are not fast enough to solve these problems.
Approach:	They build a dense retrieval based semantic search engine on scientific articles from Elsevier that generates high-quality pseudo training labels.
Outcome:	The proposed model significantly outperforms the currently deployed lexical search engine on the two test sets.

Copied to clipboard

null Govind, Saurabh Sohoney

Representation Learning

Challenge:	Existing systems for learning geolocation fail to cater to a significant fraction of addresses which are new in the system and have inaccurate or missing building level information.
Approach:	They propose a framework to resolve addresses to a shallower granularity termed neighbourhood . they propose 'deep metric learning' model to encode geospatial semantics in address embeddings .
Outcome:	The proposed framework reduces delivery defects and delivery defects in India and the United Arab Emirates.

Meta-learning Pathologies from Radiology Reports using Variance Aware Prototypical Networks (2022.emnlp-industry)

Copied to clipboard

Arijit Sehanobish, Kawshik Kannan, Nabila Abraham, Anasuya Das, Benjamin Odry

Named Entity Recognition in Industrial Tables using Tabular Language Models (2022.emnlp-industry)

Challenge:	Existing methods for fewshot learning require a large number of in-domain labeled examples for fine tuning.
Approach:	They propose to extend the Prototypical Networks for few-shot text classification by replacing Gaussian class prototypes with a regularization term that encourages the examples to be clustered near the appropriate class centroids.
Outcome:	The proposed method outperforms baselines on 13 public and 4 internal datasets and detects potential out-of-distribution (OOD) data points during deployment.

Copied to clipboard

Aneta Koleva, Martin Ringsquandl, Mark Buckley, Rakeb Hasan, Volker Tresp

Reinforced Question Rewriting for Conversational Question Answering (2022.emnlp-industry)

Challenge:	Table transformers are used for encoding tabular data but are not yet used in industrial applications.
Approach:	They propose a dedicated table data augmentation strategy based on available domain-specific knowledge graphs to enhance the performance of transformer-based models.
Outcome:	The proposed model outperforms baseline models and its inductive bias is vital for convergence of transformer-based models.

Copied to clipboard

Zhiyu Chen, Jie Zhao, Anjie Fang, Besnik Fetahu, Oleg Rokhlenko, Shervin Malmasi

Improving Large-Scale Conversational Assistants using Model Interpretation based Training Sample Selection (2022.emnlp-industry)

Challenge:	Existing approaches to CQA involve training new models from scratch . existing approaches are expensive and often not feasible .
Approach:	They propose to use QA feedback to supervise the rewriting model with reinforcement learning.
Outcome:	The proposed model can improve QA performance over baselines for extractive and retrieval QA.

Copied to clipboard

Stefan Schroedl, Manoj Kumar, Kiana Hajebi, Morteza Ziyadi, Sriram Venkatapathy, Anil Ramakrishna, Rahul Gupta, Pradeep Natarajan

Improving Precancerous Case Characterization via Transformer-based Ensemble Learning (2022.emnlp-industry)

Challenge:	Large-scale, voice-based conversational assistants process each utterance through a multi-stage pipeline that includes wakeword detection, automatic speech recognition (ASR), natural language understanding (NLU), entity resolution, and textto-speech.
Approach:	They propose a method to identify customer implicitly satisfied with Alexa's responses by leveraging interpretations of model behavior.
Outcome:	The proposed approach produces statistically significant improvements in both offline and online tests.

Copied to clipboard

Yizhen Zhong, Jiajie Xiao, Thomas Vetterli, Mahan Matin, Ellen Loo, Jimmy Lin, Richard Bourgon, Ofer Shapira

Developing Prefix-Tuning Models for Hierarchical Text Classification (2022.emnlp-industry)

Challenge:	Application of natural language processing (NLP) to cancer pathology reports has been focused on detecting cancer cases, ignoring precancerous cases.
Approach:	They developed transformer-based deep neural network NLP models to perform the CRC phenotyping with the goal of extracting precancerous lesion attributes and distinguishing cancer and precancirous cases.
Outcome:	The proposed model achieves 0.914 macro-F1 scores for classifying patients into negative, non-advanced adenoma, advanced adénoma and CRC.

Copied to clipboard

Lei Chen, Houwei Chou, Xiaodan Zhu

PAIGE: Personalized Adaptive Interactions Graph Encoder for Query Rewriting in Dialogue Systems (2022.emnlp-industry)

Challenge:	Hierarchical text classification (HTC) is a key task in many industrial applications. Pre-trained Language Models (PLMs) have become dominant for most natural language processing (NLP) tasks.
Approach:	They investigate how prefix tuning can improve hierarchical text classification . prefix-tuning model only needs less than 1% of parameters to achieve performance .
Outcome:	The proposed model can achieve comparable performance to regular full fine-tuning.

Copied to clipboard

Daniel Biś, Saurabh Gupta, Jie Hao, Xing Fan, Chenlei Guo

Fast Vocabulary Transfer for Language Model Compression (2022.emnlp-industry)

Challenge:	Existing methods to fix faulty queries are limited in their ability to fix them.
Approach:	They propose a Personalized Adaptive Interactions Graph Encoder that integrates user's affinities and query semantics to refine utterance embeddings.
Outcome:	The proposed Query Rewriting (QR) techniques improve the rewrite accuracy of state-of-the-art baselines by 12.5–17.5% while having nearly ten times fewer parameters.

Copied to clipboard

Leonidas Gee, Andrea Zugarini, Leonardo Rigutini, Paolo Torroni

Multimodal Context Carryover (2022.emnlp-industry)

Challenge:	Existing methods to reduce model size and size are expensive and inefficient for some applications.
Approach:	They propose a method that relies on vocabulary transfer to reduce model size and inference time while compromising on performance.
Outcome:	The proposed method reduces model size and inference time while compromising on performance.

Copied to clipboard

Prashan Wanigasekara, Nalin Gupta, Fan Yang, Emre Barut, Zeynab Raeesy, Kechen Qin, Stephen Rawls, Xinyue Liu, Chengwei Su, Spurthi Sandiri

Distilling Multilingual Transformers into CNNs for Scalable Intent Classification (2022.emnlp-industry)

Challenge:	Existing voice-only dialogue systems lack multimodality support, which can lead to costly system redesigns.
Approach:	They propose to augment existing voice-only dialogue systems with additional multimodal components to facilitate quick delivery of visual modality support with minimal changes.
Outcome:	The proposed framework improves visual modality support with minimal changes on an in-house multi-modal visual navigation data set.

Copied to clipboard

Besnik Fetahu, Akash Veeragouni, Oleg Rokhlenko, Shervin Malmasi

Bringing the State-of-the-Art to Customers: A Neural Agent Assistant Framework for Customer Service Support (2022.emnlp-industry)

Challenge:	Existing multilingual models for voice assistants are limited by their prohibitive inference time and limited performance.
Approach:	They propose to distill and deploy multilingual Transformer models for voice assistants using a teacher-student framework that uses teacher-trained models to supervise student model training.
Outcome:	The proposed model outperforms a teacher model trained on unlabelled data and achieves equivalent performance.

Copied to clipboard

Stephen Obadinma, Faiza Khan Khattak, Shirley Wang, Tania Sidhorn, Elaine Lau, Sean Robertson, Jingcheng Niu, Winnie Au, Alif Munim, Karthik Raja Kalaiselvi Bhaskar

Challenge:	Creating agent assistants that can help improve customer service support requires inputs from industry users and their customers as well as knowledge of state-of-the-art natural language processing (NLP) technology.
Approach:	They propose to combine expertise from academia and industry to build task/domain-specific Neural Agent Assistants with three high-level components for: (1) Intent Identification, (2) Context Retrieval, and (3) Response Generation.
Outcome:	The proposed framework is based on three case studies of industry partners who successfully adapt the framework to their unique challenges.

Zero-Shot Dynamic Quantization for Transformer Inference (2022.emnlp-industry)

Copied to clipboard

Yousef El-kurdi, Jerry Quinn, Avi Sil

Fact Checking Machine Generated Text with Dependency Trees (2022.emnlp-industry)

Challenge:	Existing methods for quantizing models require calibration or modification of parameters . run-time inference of such large models is costly due to large computational requirements .
Approach:	They propose a run-time method for quantizing BERT-like models to 8-bit integers . they demonstrate that the method can be used on many NLP tasks without calibration steps .
Outcome:	The proposed method reduces the accuracy loss associated with quantizing BERT-like models to 8-bit integers.

Copied to clipboard

Alex Estes, Nikhita Vedula, Marcus Collins, Matt Cecil, Oleg Rokhlenko

Structured Data in NLP

Challenge:	Recent work has noted the benefits of natural language text generated by NLG systems over fixed templates.
Approach:	They propose a method that checks factuality of input text based on structured knowledge patterns and dependency relations with respect to the input text.
Outcome:	The proposed technique outperforms state-of-the-art techniques in this special, but important case.

Prototype-Representations for Training Data Filtering in Weakly-Supervised Information Extraction (2022.emnlp-industry)

Copied to clipboard

Nasser Zalmout, Xian Li

CGF: Constrained Generation Framework for Query Rewriting in Conversational AI (2022.emnlp-industry)

Challenge:	Weak supervision and data programming are powerful tools to support information extraction models.
Approach:	They propose a prototype-based method to denoise weakly supervised training data . they use a model to model the correct contexts for a given target value .
Outcome:	The proposed method achieves 9% accuracy gain in attribute value extraction in e-commerce websites.

Copied to clipboard

Jie Hao, Yang Liu, Xing Fan, Saurabh Gupta, Saleh Soltan, Rakesh Chada, Pradeep Natarajan, Chenlei Guo, Gokhan Tur

Paraphrasing

Challenge:	Large-scale conversational AI agents such as Alexa, Siri and Google Assistant help millions of users to perform a lot of tasks.
Approach:	They propose a Constrained Generation Framework for query rewriting at global and personalized levels.
Outcome:	The proposed framework significantly boosts the query rewriting performance.

Entity-level Sentiment Analysis in Contact Center Telephone Conversations (2022.emnlp-industry)

Copied to clipboard

Xue-yong Fu, Cheng Chen, Md Tahmid Rahman Laskar, Shayna Gardiner, Pooja Hiranandani, Shashi Bhushan Tn

QUILL: Query Intent with Large Language Models using Retrieval Augmentation and Multi-stage Distillation (2022.emnlp-industry)

Challenge:	Entity-level sentiment analysis is useful in a business context to understand user emotions towards certain entities.
Approach:	They propose to use a model that predicts the sentiment about entities mentioned in a given text to build an entity-level sentiment analysis system that analyzes English telephone conversation transcripts.
Outcome:	The proposed system analyzes English telephone conversation transcripts to provide business insight.

Copied to clipboard

Krishna Srinivasan, Karthik Raman, Anupam Samanta, Lingrui Liao, Luca Bertelli, Michael Bendersky

Distinguish Sense from Nonsense: Out-of-Scope Detection for Virtual Assistants (2022.emnlp-industry)

Challenge:	Large Language Models (LLMs) have shown impressive results on a variety of text understanding tasks.
Approach:	They propose a two-stage distillation approach that allows retrieval augmentation to be carried over without the increased compute associated with it.
Outcome:	The proposed approach can carry over the gains of retrieval augmentation without suffering the increased compute typically associated with it.

Copied to clipboard

Cheng Qian, Haode Qi, Gengyu Wang, Ladislav Kunc, Saloni Potdar

Dialogue Systems & Conversational Agents

Challenge:	Out of Scope (OOS) detection is a problem with chatbots that cannot make sense of a query . a real-world solution to this problem is to identify out-of-domain queries .
Approach:	They propose a simple yet effective OOS detection method that outperforms standard methods . they propose analyzing data from an enterprise virtual assistant platform to test the method .
Outcome:	The proposed method outperforms standard OOS detection methods in a real-world deployment of virtual assistants.

PLATO-Ad: A Unified Advertisement Text Generation Framework with Multi-Task Prompt Learning (2022.emnlp-industry)

Copied to clipboard

Zeyang Lei, Chao Zhang, Xinchao Xu, Wenquan Wu, Zheng-yu Niu, Hua Wu, Haifeng Wang, Yi Yang, Shuanglong Li

Dense Feature Memory Augmented Transformers for COVID-19 Vaccination Search Classification (2022.emnlp-industry)

Challenge:	Online advertisement text generation models have achieved remarkable success in generating high-quality text ads, but some challenges remain, such as low-resource scenarios and training efficiency for multiple ad tasks.
Approach:	They propose a unified text ad generation framework with multi-task prompt learning to tackle low-resource ade generation problem and a multi-step prompt learning mechanism to efficiently solve multiple aed generation tasks.
Outcome:	The proposed framework outperforms the state-of-the-art on offline and online metrics.

Copied to clipboard

Jai Gupta, Yi Tay, Chaitanya Kamath, Vinh Tran, Donald Metzler, Shailesh Bavadekar, Mimi Sun, Evgeniy Gabrilovich

Full-Stack Information Extraction System for Cybersecurity Intelligence (2022.emnlp-industry)

Challenge:	a new method for classification of COVID-19 vaccination related search queries is proposed . the proposed method uses pretrained Transformers and dense features to generate search insights .
Approach:	They propose a machine learning model that detects COVID-19 vaccination related search queries . they use pretrained Transformers to consider dense features as memory tokens that the model can attend to .
Outcome:	The proposed model improves the Vaccine Search Insights task by +15% . the proposed model uses pretrained Transformers and traditional dense features .

Copied to clipboard

Youngja Park, Taesung Lee

Deploying Unified BERT Moderation Model for E-Commerce Reviews (2022.emnlp-industry)

Challenge:	a rapid increase in cyberattacks and attacks pose enormous challenges to security analysts.
Approach:	They propose a full-stack information extraction system for the cybersecurity domain that extracts 26 entity types, 20 fixed relations and the temporal information of the relations.
Outcome:	The proposed system can extract 26 entity types, 20 fixed rela and temporal information of relations.

Copied to clipboard

Ravindra Nayak, Nikesh Garera

SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrieval (2022.emnlp-industry)

Challenge:	Product reviews and ratings are an integral part of the shopping experience to build trust among users.
Approach:	They propose a system and a model for automated moderation of user-generated content that can be pre-trained with monolingual data to cope with the domain gap.
Outcome:	The proposed model achieves 4.78% increase in F1 score with less labelled data and 2.57% increase in score on the review data compared to the publicly available models.

Copied to clipboard

Kun Zhou, Yeyun Gong, Xiao Liu, Wayne Xin Zhao, Yelong Shen, Anlei Dong, Jingwen Lu, Rangan Majumder, Ji-rong Wen, Nan Duan

Revisiting and Advancing Chinese Natural Language Understanding with Accelerated Heterogeneous Knowledge Pre-training (2022.emnlp-industry)

Challenge:	Existing methods for sapping negatives from large document pool suffer from the uninformative or false negative problem.
Approach:	They propose a method to sample negatives from a large document pool using a new sampling probability distribution.
Outcome:	The proposed method can be used to sample more ambiguous negatives on four public and one industry datasets.

Copied to clipboard

Taolin Zhang, Junwei Dong, Jianing Wang, Chengyu Wang, Ang Wang, Yinghui Liu, Jun Huang, Yong Li, Xiaofeng He

A Stacking-based Efficient Method for Toxic Language Detection on Live Streaming Chat (2022.emnlp-industry)

Challenge:	Existing knowledge-enhanced pre-trained language models (KEPLMs) can capture internal knowledge, but can't understand external background knowledge.
Approach:	They propose to use Chinese knowledge-enhanced pre-trained language models to improve context-aware representations via learning from structured relations in knowledge bases.
Outcome:	Experiments show that Chinese knowledge-enhanced pre-trained language models outperform strong baselines over various benchmark NLP tasks and in different model sizes.

Copied to clipboard

Yuto Oikawa, Yuki Nakayama, Koji Murakami

Green & Sustainable NLP

Challenge:	Existing methods for toxic language detection are based on deep learning, but they are not scalable considering inference speed and computational resources.
Approach:	They propose a method for toxic language detection that is aware of real-world scenarios by partial stacking partial stacks that feeds initial results with low confidence to meta-classifier.
Outcome:	The proposed method achieves faster inference speed than BERT-based models with comparable performance.

End-to-End Speech to Intent Prediction to improve E-commerce Customer Support Voicebot in Hindi and English (2022.emnlp-industry)

Copied to clipboard

Abhinav Goyal, Anupam Singh, Nikesh Garera

PILE: Pairwise Iterative Logits Ensemble for Multi-Teacher Labeled Distillation (2022.emnlp-industry)

Challenge:	Experimental results show that an E2E S2I model outperforms a conventional pipeline by 27% on the F1 score . multi-component pipelines require large annotated datasets, have higher latency, and have complex deployment.
Approach:	They propose an end-to-end (E2E) S2I model that uses pre-trained automatic speech recognition with slight modification and fine-tuning on small annotated datasets.
Outcome:	The proposed model outperforms a conventional pipeline on an annotated dataset by 27% on the F1 score.

Copied to clipboard

Lianshang Cai, Linhao Zhang, Dehong Ma, Jun Fan, Daiting Shi, Yi Wu, Zhicong Cheng, Simiu Gu, Dawei Yin

A Comprehensive Evaluation of Biomedical Entity-centric Search (2022.emnlp-industry)

Challenge:	Pre-trained language models have been a key part of ranking systems . knowledge distillation is widely used to maintain high performance while keeping efficient computations.
Approach:	They propose an algorithm to combine knowledge from multi-teachers and label information to achieve competitive performance in offline and online experiments.
Outcome:	The proposed method has been deployed in a real-world commercial search system.

Copied to clipboard

Elena Tutubalina, Zulfat Miftahutdinov, Vladimir Muravlev, Anastasia Shneyderman

Domain Adaptation of Machine Translation with Crowdworkers (2022.emnlp-industry)

Challenge:	a novel algorithm for information retrieval from biomedical abstracts is used to identify entities.
Approach:	They perform a fine-grained evaluation of a BERT-based biomedical search engine . they use manually annotated PubMed abstracts and off-she-shelf Elasticsearch .
Outcome:	The proposed system performs better for disease and gene search queries than other systems.

Copied to clipboard

Makoto Morishita, Jun Suzuki, Masaaki Nagata

Text Generation

Challenge:	Existing methods for machine translation work well when no in-domain data are available.
Approach:	They propose a framework that efficiently and effectively collects parallel sentences in a target domain from the web with the help of crowdworkers.
Outcome:	The proposed method improves the BLEU scores by an average of +7.8 points compared to a general-purpose translation model.

Biomedical NER for the Enterprise with Distillated BERN2 and the Kazu Framework (2022.emnlp-industry)

Copied to clipboard

Wonjin Yoon, Richard Jackson, Elliot Ford, Vladimir Poroshin, Jaewoo Kang

Large-scale Machine Translation for Indian Languages in E-commerce under Low Resource Constraints (2022.emnlp-industry)

Challenge:	a low overall error rate is unavoidable in the field of bioNLP, argues a new study.
Approach:	They propose a highly extensible open source framework that supports BioNLP for the pharmaceutical sector.
Outcome:	The proposed framework is extensible and scalable and can support BioNLP for the pharmaceutical sector.

Copied to clipboard

Amey Patil, Nikesh Garera

Machine Translation

Challenge:	We have deployed reliable and precise large-scale machine translation systems for several Indian regional languages.
Approach:	They develop a structured model development pipeline as a closed feedback loop with external manual feedback through an Active Learning component.
Outcome:	The proposed model improves over iterations for English to Hindi and for other languages.

Topic Modeling by Clustering Language Model Embeddings: Human Validation on an Industry Dataset (2022.emnlp-industry)

Copied to clipboard

Anton Eklund, Mona Forsman