Papers by Eugene Agichtein
VoiSeR: A New Benchmark for Voice-Based Search Refinement (2021.eacl-main)
Copied to clipboard
| Challenge: | a new study shows that voice-based search systems are challenging to support in the context of the user intent of voice searches . support for voice-driven search, exploration, and refinement is a fundamental aspect of voice assistants . |
| Approach: | They propose to use crowdsourcing to collect voice-based search refinements . they use 10,000 search refinement utterances to annotate a search intent . |
| Outcome: | The proposed dataset shows that voice-based search refinements can support most common tasks . the study shows that the proposed dataset can support research in conversational query understanding . |
You Sound Like Someone Who Watches Drama Movies: Towards Predicting Movie Preferences from Conversational Interactions (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for conversational recommendation include collaborative filtering, content-based filtering and user reviews. |
| Approach: | They propose to map a conversational user to most similar external reviewers, whose preferences are known, and adapt collaborative filtering techniques to estimate the current user’s preferences for new movies. |
| Outcome: | The proposed method can improve the accuracy of predicting user ratings for new movies by exploiting conversation content and external data. |
DUQGen: Effective Unsupervised Domain Adaptation of Neural Rankers by Diversifying Synthetic Query Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | State-of-the-art rankers pre-trained on large task-specific training data such as MS-MARCO exhibit strong performance on various ranking tasks without domain adaptation, also called zero-shot. |
| Approach: | They propose a method to generate unsupervised domain adaptation for ranking using large-scale task-specific training data such as MS-MARCO and Wikipedia retrieval. |
| Outcome: | The proposed method outperforms all zero-shot baselines and significantly outperfies the SOTA baselines on 16 out of 18 datasets, for an average of 4% relative improvement across all datasets. |
Identifying Helpful Sentences in Product Reviews (2021.naacl-main)
Copied to clipboard
| Challenge: | a key advantage of online shopping is the ability to read what other customers are saying about products of interest. |
| Approach: | They propose a task to extract a representative helpful sentence from reviews . they collect a dataset in english and use crowd-sourcing to test their model . |
| Outcome: | The proposed model outperforms baselines in a crowd-sourced model of representative helpful sentences from product reviews. |
Wizard of Tasks: A Novel Conversational Dataset for Solving Real-World Tasks in Conversational Settings (2022.coling-1)
Copied to clipboard
Jason Ingyu Choi, Saar Kuzi, Nikhita Vedula, Jie Zhao, Giuseppe Castellucci, Marcus Collins, Shervin Malmasi, Oleg Rokhlenko, Eugene Agichtein
| Challenge: | Existing Conversational Task Assistants fail to provide a comprehensive natural conversation that includes search, context-aware QA, step-by-step instructions. |
| Approach: | They present a corpus of conversations in two domains: cooking and home improvement . they crowd-sourced 549 conversations with an asynchronous Wizard-of-Oz setup . |
| Outcome: | The proposed model performs well in both Intent Classification and Abstractive Question Answering tasks, but the performance is poor on AQA tasks. |
Generative Product Recommendations for Implicit Superlative Queries (2025.naacl-srw)
Copied to clipboard
| Challenge: | Existing retrieval and ranking systems struggle with implicit superlative queries . lack of explicit attribute mentions and complexity of the query complicates ranking . |
| Approach: | They propose a four-point schema for annotating the best product candidates for superlative queries . they propose pointwise, deliberated pointwise and pairwise methods to analyze the results . |
| Outcome: | The proposed schema can be used to rank products with implicit attributes and reason over them. |
QueryExplorer: An Interactive Query Generation Assistant for Search and Exploration (2024.naacl-demo)
Copied to clipboard
| Challenge: | Formulating effective search queries can be a daunting task for users when they lack expertise in a specific domain or are not proficient in the language of the content. |
| Approach: | QueryExplorer is an interactive query generation, reformulation, and retrieval interface with support for Hug-gingFace generation models and PyTerrier’sretrieval pipelines and datasets. |
| Outcome: | QueryExplorer is an interactive query generation, reformulation, and retrieval interface with support for Hug-gingFace generation models and PyTerrier’sretrieval pipelines and datasets, and extensivelogging of human feedback. |
ConQRet: A New Benchmark for Fine-Grained Automatic Evaluation of Retrieval Augmented Computational Argumentation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for evaluating RAArg are costly and lack long, complex arguments and real-world evidence. |
| Approach: | They propose to use multiple fine-grained LLM judges to evaluate RAArg using a new benchmark that features long and complex human-authored arguments on debated topics. |
| Outcome: | The proposed methods provide better and more interpretable assessments than traditional single-score metrics and even previously reported human crowdsourcing. |