Papers by Nicholas Monath
Analysis of Plan-based Retrieval for Grounded Text Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large, parametric language models (LLMs) produce fluent text for many applications . hallucinations are generation of text that is factually correct and semantically plausible . |
| Approach: | They propose to use learning-tuned LLMs to infuse models with retrieval mechanisms to reduce hallucinations. |
| Outcome: | The proposed approach reduces the frequency of hallucinations by reducing the coverage of relevant facts and generating more informative responses while providing higher attribution rates. |
Optimal Transport-based Alignment of Learned Character Representations for String Similarity (P19-1)
Copied to clipboard
| Challenge: | String similarity models are crucial for record linkage, data integration, search and entity resolution systems. |
| Approach: | They propose a model that encodes the characters of each string, aligns the encodings using Sinkhorn Iteration and scores the alignment with a convolutional neural network. |
| Outcome: | The proposed model outperforms state-of-the-art and classical similarity models on four of the five datasets and improves performance by applying it to cross-document coreference. |
Autoregressive Structured Prediction with Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent years have seen a paradigm shift in NLP towards using pretrained language models for a wide range of tasks. |
| Approach: | They propose to model structures as sequences of actions in autoregressive manner with PLMs . their approach allows in-structure dependencies to be learned without any loss . |
| Outcome: | The proposed approach achieves state-of-the-art on all structured prediction tasks. |
Efficient k-NN Search with Cross-Encoders using Adaptive Multi-Round CUR Decomposition (2023.findings-emnlp)
Copied to clipboard
| Challenge: | ANNCUR uses a cross-encoder only to perform k-NN search, but the approximation of the distances is often detrimental to the retrieval of top-k items. |
| Approach: | They propose a method that minimizes approximation error for k-nearest neighbor searches . they propose to use a cross-encoder only to perform k NN search . |
| Outcome: | The proposed method reduces approximation error for top-k neighbors by up to 70% . iteratively performs k-NN search using the available anchors, then adds them to the next set . |
PRISM: Efficient Long-Range Reasoning With Short-Context LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing solutions to long-range language tasks require large compute budgets and complex task-specific design choices. |
| Approach: | They propose an in-context method that uses structured schemas to generate short-contemporary outputs. |
| Outcome: | a new in-context method outperforms baselines on diverse tasks with 4x shorter contexts . it scales down to tiny contexts without increasing costs or sacrificing quality . |
Scaling Within Document Coreference to Long Texts (2021.findings-acl)
Copied to clipboard
Raghuveer Thirukovalluru, Nicholas Monath, Kumar Shridhar, Manzil Zaheer, Mrinmaya Sachan, Andrew McCallum
| Challenge: | Existing end-to-end coreference resolution models use expensive span representations and antecedent prediction mechanisms. |
| Approach: | They propose an approximation to end-to-end coreference resolution models which scales gracefully to documents of any length. |
| Outcome: | The proposed model reduces training and inference time and memory costs compared to current models with minimal loss in accuracy. |
Unsupervised Opinion Summarization Using Approximate Geodesics (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for opinion summarization are limited due to the scarcity of data. |
| Approach: | They propose a system to perform unsupervised extractive opinion summarization using a dictionary-based representation learning model that generates topical representations of texts. |
| Outcome: | The proposed system achieves strong performance on three opinion summarization datasets. |
Entity Linking via Explicit Mention-Mention Coreference Modeling (2022.naacl-main)
Copied to clipboard
| Challenge: | Using a learning approach for entity mentions is a key component of modern entity linking systems for both candidate generation and making linking predictions. |
| Approach: | They propose a training approach that builds minimum spanning arborescences over mentions and entities to explicitly model mention coreference relationships. |
| Outcome: | The proposed approach improves candidate generation recall and link accuracy on the biomedical dataset and on MedMentions, setting a new SOTA result in linking accuracy. |
Longtonotes: OntoNotes with Longer Coreference Chains (2023.findings-eacl)
Copied to clipboard
Kumar Shridhar, Nicholas Monath, Raghuveer Thirukovalluru, Alessandro Stolfo, Manzil Zaheer, Andrew McCallum, Mrinmaya Sachan
| Challenge: | Using Ontonotes, documents in certain genres were split into smaller parts for ease of annotation. |
| Approach: | They propose to merge annotations from documents split into smaller parts in Ontonotes for ease of annotation. |
| Outcome: | The proposed corpus restores documents to their original form, revealing dramatic increases in length in certain genres. |
Clustering-based Inference for Biomedical Entity Linking (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to linking entities ignore relationships between entities in biomedical knowledge bases. |
| Approach: | They propose a model which can link mentions of unseen entities using learned representations of entities. |
| Outcome: | The proposed model improves on the largest publicly available biomedical dataset by 3.0 points of accuracy and 2.3 points of reliability. |
SIKeD: Self-guided Iterative Knowledge Distillation for Mathematical Reasoning (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) can generate intermediate reasoning process for multistep reasoning tasks. |
| Approach: | They propose a distillation method that teaches the model to approach a task using different strategies and the model uses its self-generated on-policy outputs to choose the most suitable strategy. |
| Outcome: | The proposed method significantly outperforms distillation techniques on large models of different sizes. |
Efficient Nearest Neighbor Search for Cross-Encoder Models using Matrix Factorization (2022.emnlp-main)
Copied to clipboard
| Challenge: | Efficient k-nearest neighbor search is a fundamental task, foundational for many problems in NLP. |
| Approach: | They propose an approach that avoids the use of a dual-encoder for retrieval, relying solely on the cross-encoding model. |
| Outcome: | Empirically, for k > 10, our approach provides test-time recall-vs-computational cost trade-offs superior to the current widely-used methods that re-rank items retrieved using a dual-encoder or TF-IDF. |
Probabilistic Case-based Reasoning for Open-World Knowledge Graph Completion (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for learning non-parametric representations of entities and relations are based on tensor factorization or sophisticated neural approaches. |
| Approach: | They propose a case-based reasoning system that retrieves ‘cases’ that are similar to the given problem and then stores them in its parameters. |
| Outcome: | The proposed model outperforms state-of-the-art methods on several benchmark datasets and is non-parametric and grows dynamically as new entities and relations arrive in the KB. |
Sequence Reducible Holdout Loss for Language Model Pretraining (2024.lrec-main)
Copied to clipboard
| Challenge: | Data selection techniques have shown empirical benefits in reducing the number of gradient steps to train neural models. |
| Approach: | They propose to modify an existing data selection technique to adapt it to the sequence losses typical in language modeling. |
| Outcome: | The proposed technique reduces the number of steps required to train neural models by 4.3% and improves generalization ability on out of domain datasets. |