Papers by Manzil Zaheer
Analysis of Plan-based Retrieval for Grounded Text Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large, parametric language models (LLMs) produce fluent text for many applications . hallucinations are generation of text that is factually correct and semantically plausible . |
| Approach: | They propose to use learning-tuned LLMs to infuse models with retrieval mechanisms to reduce hallucinations. |
| Outcome: | The proposed approach reduces the frequency of hallucinations by reducing the coverage of relevant facts and generating more informative responses while providing higher attribution rates. |
Investigating the Working of Text Classifiers (C18-1)
Copied to clipboard
| Challenge: | Text classification is one of the most widely studied tasks in natural language processing. |
| Approach: | They propose to use large multilayer neural network models to compose meaning of sentences . they propose to disincentivize focusing on key lexicons to improve classification accuracy . |
| Outcome: | The proposed models learn to compose the meaning of the sentences or focus on key lexicons for classifying the document. |
Questions Are All You Need to Train a Dense Passage Retriever (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing methods for dense retrieval require large supervised datasets with custom hard-negative mining and denoising of positive examples. |
| Approach: | They propose a new corpus-level autoencoding approach for training dense retrieval models that does not require labeled training data. |
| Outcome: | The proposed method matches or surpasses strong supervised performance levels on multiple QA benchmarks with no labeled training data or task-specific losses. |
Efficient k-NN Search with Cross-Encoders using Adaptive Multi-Round CUR Decomposition (2023.findings-emnlp)
Copied to clipboard
| Challenge: | ANNCUR uses a cross-encoder only to perform k-NN search, but the approximation of the distances is often detrimental to the retrieval of top-k items. |
| Approach: | They propose a method that minimizes approximation error for k-nearest neighbor searches . they propose to use a cross-encoder only to perform k NN search . |
| Outcome: | The proposed method reduces approximation error for top-k neighbors by up to 70% . iteratively performs k-NN search using the available anchors, then adds them to the next set . |
Can Public Large Language Models Help Private Cross-device Federated Learning? (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have shown that public data can be used to improve privacy-utility trade-offs for large and small language models. |
| Approach: | They propose to use large-scale public data to help differentially private FL training . they propose a distribution matching algorithm with theoretical grounding to sample public data close to private data distribution . |
| Outcome: | The proposed method is efficient and effective for training private models by taking advantage of public data. |
Differentiable Open-Ended Commonsense Reasoning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing commonsense reasoning models work by scoring a question-candidate pair, but new approaches are needed to answer multiple-choice questions. |
| Approach: | They propose to use a corpus of commonsense facts to answer a commonsensical question without any pre-defined choices as a resource. |
| Outcome: | The proposed model outperforms baseline methods by a large margin in the open-ended commonsense reasoning task. |
Scaling Within Document Coreference to Long Texts (2021.findings-acl)
Copied to clipboard
Raghuveer Thirukovalluru, Nicholas Monath, Kumar Shridhar, Manzil Zaheer, Mrinmaya Sachan, Andrew McCallum
| Challenge: | Existing end-to-end coreference resolution models use expensive span representations and antecedent prediction mechanisms. |
| Approach: | They propose an approximation to end-to-end coreference resolution models which scales gracefully to documents of any length. |
| Outcome: | The proposed model reduces training and inference time and memory costs compared to current models with minimal loss in accuracy. |
Multi-step Entity-centric Information Retrieval for Multi-Hop Question Answering (D19-58)
Copied to clipboard
Rajarshi Das, Ameya Godbole, Dilip Kavarthapu, Zhiyu Gong, Abhishek Singhal, Mo Yu, Xiaoxiao Guo, Tian Gao, Hamed Zamani, Manzil Zaheer, Andrew McCallum
| Challenge: | Multi-hop question answering (QA) requires an information retrieval system that can find multiple supporting evidence needed to answer the question. |
| Approach: | They propose a technique that uses information of entities present in the initial retrieved evidence to learn to ‘hop’ onto other relevant evidence. |
| Outcome: | The proposed method boosts retrieval performance on a multi-hop question answering dataset with 5 million Wikipedia paragraphs and a model without training increases its performance by 10.59 F1. |
Longtonotes: OntoNotes with Longer Coreference Chains (2023.findings-eacl)
Copied to clipboard
Kumar Shridhar, Nicholas Monath, Raghuveer Thirukovalluru, Alessandro Stolfo, Manzil Zaheer, Andrew McCallum, Mrinmaya Sachan
| Challenge: | Using Ontonotes, documents in certain genres were split into smaller parts for ease of annotation. |
| Approach: | They propose to merge annotations from documents split into smaller parts in Ontonotes for ease of annotation. |
| Outcome: | The proposed corpus restores documents to their original form, revealing dramatic increases in length in certain genres. |
Case-based Reasoning for Natural Language Queries over Knowledge Bases (2021.emnlp-main)
Copied to clipboard
Rajarshi Das, Manzil Zaheer, Dung Thai, Ameya Godbole, Ethan Perez, Jay Yoon Lee, Lizhen Tan, Lazaros Polymenakos, Andrew McCallum
| Challenge: | Using human-labeled examples, case-based reasoning can solve complex problems from scratch . case-Based reasoning is a paradigm that is used to solve complex problem . |
| Approach: | They propose a neuro-symbolic CBR approach for question answering over large knowledge bases. |
| Outcome: | The proposed approach outperforms the current state of the art on a CWQ dataset by 11% on accuracy. |
Machine Reading Comprehension using Case-based Reasoning (2023.findings-emnlp)
Copied to clipboard
Dung Thai, Dhruv Agarwal, Mudit Chaudhary, Wenlong Zhao, Rajarshi Das, Jay-Yoon Lee, Hannaneh Hajishirzi, Manzil Zaheer, Andrew McCallum
| Challenge: | Current state-of-the-art machine readers do not support case-based reasoning . |
| Approach: | They propose a method that extracts a set of similar cases from a nonparametric memory and then predicts an answer by selecting the span in the test context that is most similar to the contextualized representations of answers. |
| Outcome: | The proposed method outperforms baselines on NaturalQuestions and NewsQA by 11.5 and 8.4 EM. |
Efficient Nearest Neighbor Search for Cross-Encoder Models using Matrix Factorization (2022.emnlp-main)
Copied to clipboard
| Challenge: | Efficient k-nearest neighbor search is a fundamental task, foundational for many problems in NLP. |
| Approach: | They propose an approach that avoids the use of a dual-encoder for retrieval, relying solely on the cross-encoding model. |
| Outcome: | Empirically, for k > 10, our approach provides test-time recall-vs-computational cost trade-offs superior to the current widely-used methods that re-rank items retrieved using a dual-encoder or TF-IDF. |
Large Language Models with Controllable Working Memory (2023.findings-acl)
Copied to clipboard
Daliang Li, Ankit Singh Rawat, Manzil Zaheer, Xin Wang, Michal Lukasik, Andreas Veit, Felix Yu, Sanjiv Kumar
| Challenge: | Large language models (LLMs) have led to a series of breakthroughs in natural language processing due to the massive amounts of world knowledge they memorize during pretraining. |
| Approach: | They propose a method to inject counterfactual and irrelevant contexts into standard supervised datasets to strengthen both controllability and robustness. |
| Outcome: | The proposed method improves controllability and robustness across model architectures and sizes. |
Probabilistic Case-based Reasoning for Open-World Knowledge Graph Completion (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for learning non-parametric representations of entities and relations are based on tensor factorization or sophisticated neural approaches. |
| Approach: | They propose a case-based reasoning system that retrieves ‘cases’ that are similar to the given problem and then stores them in its parameters. |
| Outcome: | The proposed model outperforms state-of-the-art methods on several benchmark datasets and is non-parametric and grows dynamically as new entities and relations arrive in the KB. |
Chains-of-Reasoning at TextGraphs 2019 Shared Task: Reasoning over Chains of Facts for Explainable Multi-hop Inference (D19-53)
Copied to clipboard
| Challenge: | EMNLP 2019 shared task on 'Multi-hop Inference Explanation Regeneration' identifies chains of facts relevant to explain an answer to an elementary science examination question. |
| Approach: | They propose a system that identifies chains of facts relevant to explain an answer to an elementary science examination question. |
| Outcome: | The proposed system outperforms the second best system by 14.95 points on the mean average precision (MAP) metric. |
Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text (D18-1)
Copied to clipboard
| Challenge: | Specialized neural models have been developed for extracting answers from text alone or Knowledge Bases (KBs) alone. |
| Approach: | They propose a novel model for extracting answers from a question-specific subgraph containing text and KB entities and relations. |
| Outcome: | The proposed model outperforms existing methods in a combination of a KB and entity-linked text in QA over a large text corpus. |