Papers by Oleg Rokhlenko
VoiSeR: A New Benchmark for Voice-Based Search Refinement (2021.eacl-main)
Copied to clipboard
| Challenge: | a new study shows that voice-based search systems are challenging to support in the context of the user intent of voice searches . support for voice-driven search, exploration, and refinement is a fundamental aspect of voice assistants . |
| Approach: | They propose to use crowdsourcing to collect voice-based search refinements . they use 10,000 search refinement utterances to annotate a search intent . |
| Outcome: | The proposed dataset shows that voice-based search refinements can support most common tasks . the study shows that the proposed dataset can support research in conversational query understanding . |
Reinforced Question Rewriting for Conversational Question Answering (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Existing approaches to CQA involve training new models from scratch . existing approaches are expensive and often not feasible . |
| Approach: | They propose to use QA feedback to supervise the rewriting model with reinforcement learning. |
| Outcome: | The proposed model can improve QA performance over baselines for extractive and retrieval QA. |
Dynamic Gazetteer Integration in Multilingual Models for Cross-Lingual and Cross-Domain Named Entity Recognition (2022.naacl-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) models trained on CoNLL do not transfer well to other domains, even within the same language. |
| Approach: | They propose a token-level gating layer to augment pre-trained multilingual transformers with gazetteers containing named entities (NE) from a target language or domain. |
| Outcome: | The proposed model improves on cross-lingual transfer with an F1 score of 92.92 for English and an average of 89.43 across all languages in CoNLL. |
You Sound Like Someone Who Watches Drama Movies: Towards Predicting Movie Preferences from Conversational Interactions (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for conversational recommendation include collaborative filtering, content-based filtering and user reviews. |
| Approach: | They propose to map a conversational user to most similar external reviewers, whose preferences are known, and adapt collaborative filtering techniques to estimate the current user’s preferences for new movies. |
| Outcome: | The proposed method can improve the accuracy of predicting user ratings for new movies by exploiting conversation content and external data. |
Distilling Multilingual Transformers into CNNs for Scalable Intent Classification (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Existing multilingual models for voice assistants are limited by their prohibitive inference time and limited performance. |
| Approach: | They propose to distill and deploy multilingual Transformer models for voice assistants using a teacher-student framework that uses teacher-trained models to supervise student model training. |
| Outcome: | The proposed model outperforms a teacher model trained on unlabelled data and achieves equivalent performance. |
InstructPTS: Instruction-Tuning LLMs for Product Title Summarization (2023.emnlp-industry)
Copied to clipboard
| Challenge: | E-commerce product catalogs contain billions of items with lengthy titles . this leads to a gap between how customers refer to these unnatural titles - and how they are used . |
| Approach: | They propose a novel approach to product title summarization that uses a fine-tuned instruction strategy to train a highly accurate model. |
| Outcome: | The proposed approach can generate more accurate product title summaries with an improvement of over 14 and 8 BLEU and ROUGE points. |
MultiCoNER: A Large-scale Multilingual Dataset for Complex Named Entity Recognition (2022.coling-1)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a core task in Natural Language Processing. |
| Approach: | They present a large multilingual dataset for Named Entity Recognition that covers 3 domains across 11 languages and multilingual and code-mixing subsets. |
| Outcome: | The proposed dataset is large and multilingual, covering 11 languages and subsets. |
Answering Unanswered Questions through Semantic Reformulations in Spoken QA (2023.acl-industry)
Copied to clipboard
| Challenge: | Question Answering (QA) is a longstanding NLP task, and voice assistants like Alexa have made Spoken QA ubiquitous. |
| Approach: | They propose a model that uses linguistically-grounded operations to rewrite questions to facilitate answering. |
| Outcome: | The proposed model improves answer rates on 1M unanswered questions from a leading voice assistant. |
External Knowledge Acquisition for End-to-End Document-Oriented Dialog Systems (2023.eacl-main)
Copied to clipboard
| Challenge: | End-to-end neural models for conversational AI often assume that a response can be generated by considering only the knowledge acquired during training. |
| Approach: | They propose an architecture for document-oriented conversations with access to external knowledge sources. |
| Outcome: | The proposed architecture outperforms baseline models on the Wizard of Wikipedia dataset by 10.3% and 7.4%. |
GEMNET: Effective Gated Gazetteer Representations for Recognizing Complex Entities in Low-context Input (2021.naacl-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is difficult in real-world settings due to short texts, emerging entities, and complex entities. |
| Approach: | They propose a flexible Gazetteer Representation encoder and a Mixture-of-Experts gating network for gazetteer knowledge integration. |
| Outcome: | The proposed approach shows large gains (up to +49% F1) in recognizing difficult entities compared to baselines. |
Faithful Low-Resource Data-to-Text Generation through Cycle Training (2023.acl-long)
Copied to clipboard
| Challenge: | Methods to generate text from structured data have advanced significantly in recent years, but can fail to produce output faithful to the input data, especially on out-of-domain data. |
| Approach: | They evaluate the effectiveness of cycle training by using two models which are inverses of each other to generate text from structured data and one which generates the structured data from natural language text. |
| Outcome: | The proposed approach achieves nearly the same performance as fully supervised approaches on the WebNLG, E2E, WTQ, and WSQL datasets. |
Wizard of Shopping: Target-Oriented E-commerce Dialogue Generation with Decision Tree Branching (2025.acl-long)
Copied to clipboard
Xiangci Li, Zhiyu Chen, Jason Ingyu Choi, Nikhita Vedula, Besnik Fetahu, Oleg Rokhlenko, Shervin Malmasi
| Challenge: | Prior human-annotated CPS datasets are small in size and lack integration with real-world product search systems. |
| Approach: | They propose a method to generate target-oriented shopping conversations without human annotations by using large language models. |
| Outcome: | The proposed method achieves highly natural and coherent conversations from three shopping domains and significantly improves on human evaluations and downstream tasks. |
Wizard of Tasks: A Novel Conversational Dataset for Solving Real-World Tasks in Conversational Settings (2022.coling-1)
Copied to clipboard
Jason Ingyu Choi, Saar Kuzi, Nikhita Vedula, Jie Zhao, Giuseppe Castellucci, Marcus Collins, Shervin Malmasi, Oleg Rokhlenko, Eugene Agichtein
| Challenge: | Existing Conversational Task Assistants fail to provide a comprehensive natural conversation that includes search, context-aware QA, step-by-step instructions. |
| Approach: | They present a corpus of conversations in two domains: cooking and home improvement . they crowd-sourced 549 conversations with an asynchronous Wizard-of-Oz setup . |
| Outcome: | The proposed model performs well in both Intent Classification and Abstractive Question Answering tasks, but the performance is poor on AQA tasks. |
Fact Checking Machine Generated Text with Dependency Trees (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Recent work has noted the benefits of natural language text generated by NLG systems over fixed templates. |
| Approach: | They propose a method that checks factuality of input text based on structured knowledge patterns and dependency relations with respect to the input text. |
| Outcome: | The proposed technique outperforms state-of-the-art techniques in this special, but important case. |
MultiCoNER v2: a Large Multilingual dataset for Fine-grained and Noisy Named Entity Recognition (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a core task in Natural Language Processing. |
| Approach: | They present a dataset for fine-grained Named Entity Recognition covering 33 entity classes across 12 languages in monolingual and multilingual settings. |
| Outcome: | The proposed dataset covers 33 entity classes across 12 languages in monolingual and multilingual settings. |
CycleKQR: Unsupervised Bidirectional Keyword-Question Rewriting (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to query paraphrases are based on encoderdecoder architectures, but they do not support the two important functionalities beyond questions. |
| Approach: | They propose a keyword-question rewriting task to improve query understanding capabilities of NLU systems for all surface forms. |
| Outcome: | Empirically, we show that CycleKQR significantly improves QA performance by rewriting queries into the appropriate form while retaining the original semantic meaning of input queries. |
Enhancing Low-Resource LLMs Classification with PEFT and Synthetic Data (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) operating in 0-shot or few-shot settings achieve competitive results in Text Classification tasks. |
| Approach: | They propose to make Large Language Models (LLMs) operating in 0-shot or few-shot settings as efficient as 0- shot text classifiers by leveraging a small number of samples. |
| Outcome: | The proposed model is able to perform better on multiple datasets than existing models on 0-shot or few-shot settings. |
Follow-on Question Suggestion via Voice Hints for Voice Assistants (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Query suggestion is a standard feature of screen-based search experiences, but it is not trivial to implement in voice-based settings. |
| Approach: | They propose a task of suggesting questions with compact voice hints to allow users to ask follow-up questions. |
| Outcome: | The proposed approach is based on a dataset of 6681 input questions and human written hints and is highly linguistically motivated. |
Generate-then-Retrieve: Intent-Aware FAQ Retrieval in Product Search (2023.acl-industry)
Copied to clipboard
| Challenge: | Frequently Asked Question (FAQ) retrieval aims at retrieving question-answer pairs for a given user query. |
| Approach: | They propose to use an intent classifier to predict whether a query is looking for an FAQ and a reformulation model to rewrite the query into a natural question to improve retrieval performance. |
| Outcome: | The proposed method improves 12% on Hit@1 on retrieving ground-truth FAQs while reducing latency by 95% compared to baseline systems. |