Papers by Lingjia Tang
One Agent To Rule Them All: Towards Multi-agent Conversational AI (2022.findings-acl)
Copied to clipboard
Christopher Clarke, Joseph Peper, Karthik Krishnamurthy, Walter Talamonti, Kevin Leach, Walter Lasecki, Yiping Kang, Lingjia Tang, Jason Mars
| Challenge: | Increasing volume of conversational agents (CAs) on the market has resulted in users being burdened with learning and adopting multiple agents to accomplish their tasks. |
| Approach: | They propose a task BBAI: Black-Box Agent Integration that integrates multiple black-box CAs at scale. |
| Outcome: | The proposed system outperforms existing benchmarks in the BBAI: Black-Box Agent Integration task. |
An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction (D19-1)
Copied to clipboard
Stefan Larson, Anish Mahendran, Joseph J. Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K. Kummerfeld, Kevin Leach, Michael A. Laurenzano, Lingjia Tang, Jason Mars
| Challenge: | Task-oriented dialog systems need to know when a query falls outside their range of supported intents. |
| Approach: | They propose a dataset that includes queries that are out-of-scope and 150 intent classes over 10 domains. |
| Outcome: | The proposed dataset includes queries that are out-of-scope, i.e., queries that do not fall into any of the system’s supported intents. |
Ranking Unraveled: Recipes for LLM Rankings in Head-to-Head AI Combat (2025.acl-long)
Copied to clipboard
| Challenge: | Evaluating large language models (LLMs) is a complex task. Pairwise ranking has emerged as state-of-the-art method to evaluate human preferences. |
| Approach: | They propose to use pairwise ranking to evaluate human preferences . they propose to evaluate the robustness of ranking algorithms in LLMs . |
| Outcome: | The proposed methods are based on the principles of effective ranking and the robustness of several ranking algorithms in the context of LLMs. |
Outlier Detection for Improved Data Quality and Diversity in Dialog Systems (N19-1)
Copied to clipboard
Stefan Larson, Anish Mahendran, Andrew Lee, Jonathan K. Kummerfeld, Parker Hill, Michael A. Laurenzano, Johann Hauswald, Lingjia Tang, Jason Mars
| Challenge: | Existing methods to detect outliers in text have been neglected in NLP . outlier detection is a problem in dialog systems where text is often no more than a few sentences in length. |
| Approach: | They propose a technique that uses sentence embeddings to detect outliers in short texts using neural sentence embeds and distance-based outlier detection. |
| Outcome: | The proposed technique detects outliers in a corpus of short texts while generating highly diverse corpora that produce more robust intent classification and slot-filling models. |
TOBUGraph: Knowledge Graph-Based Retrieval for Enhanced LLM Performance Beyond RAG (2025.emnlp-industry)
Copied to clipboard
Savini Kashmira, Jayanaka L. Dantanarayana, Joshua Brodsky, Ashish Mahendra, Yiping Kang, Krisztian Flautner, Lingjia Tang, Jason Mars
| Challenge: | Retrieval-Augmented Generation (RAG) relies on query-chunk text-to-text similarity in the embedding space for retrieval, can fail to capture deeper semantic relationships across chunks, is highly sensitive to chunking strategies, and is prone to hallucinations. |
| Approach: | They propose a graph-based retrieval framework that first constructs the knowledge graph from unstructured data dynamically and automatically. |
| Outcome: | The proposed framework outperforms multiple RAG implementations in both precision and recall, significantly enhancing user experience through improved retrieval accuracy. |
Label Agnostic Pre-training for Zero-shot Text Classification (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to text classification assume a fixed set of labels . however, in real-world applications, there exists an infinite label space for describing a given text . |
| Approach: | They propose two new methods that inject aspect-level understanding into pre-trained models at train time to improve zero-shot generalization. |
| Outcome: | The proposed methods improve zero-shot generalization on a set of challenging datasets. |
Data Collection for Dialogue System: A Startup Perspective (N18-3)
Copied to clipboard
| Challenge: | Developing dialogue systems such as Apple Siri and Google Now requires high quality training data but data collection with crowdsourcing is largely an open question. |
| Approach: | They propose to use crowdsourcing to collect data for a user intent classification task in a dialogue system. |
| Outcome: | The proposed method improves the quality of the collected data and the model performance on real user queries. |