Papers by Andrew McCallum
MOLEMAN: Mention-Only Linking of Entities with a Mention Annotation Network (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to entity linking represent each entity with a single vector, but instead use a contextualized mention-encoder that learns to place similar mentions of the same entity closer in vector space than mentions from different entities. |
| Approach: | They propose an instance-based nearest neighbor approach to entity linking that allows for a contextualized mention-encoder to learn to place similar mentions of the same entity closer in vector space than mentions from different entities. |
| Outcome: | The proposed approach outperforms all other systems on two multilingual benchmarks and is simpler to train and interpretable. |
Probabilistic Embedding of Knowledge Graphs with Box Lattice Measures (P18-1)
Copied to clipboard
| Challenge: | Structured embeddings based on regions, densities, and orderings have gained popularity for their inductive bias towards the essential asymmetries inherent in problems such as image captioning. |
| Approach: | They propose a box lattice and accompanying probability measure to capture negative correlations over arbitrary concepts. |
| Outcome: | The proposed model can capture anti-correlation and even disjoint concepts while learning from and predicting calibrated uncertainty. |
Self-Supervised Meta-Learning for Few-Shot Natural Language Classification Tasks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for supervised meta-learning require many training tasks to generalize . cloze-style objectives can be used to generate a large, rich, meta-training task distribution from unlabeled text. |
| Approach: | They propose a self-supervised approach to generate a large, rich, meta-learning task distribution from unlabeled text. |
| Outcome: | The proposed approach generates a large, rich, meta-learning task distribution from unlabeled text. |
Analysis of Plan-based Retrieval for Grounded Text Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large, parametric language models (LLMs) produce fluent text for many applications . hallucinations are generation of text that is factually correct and semantically plausible . |
| Approach: | They propose to use learning-tuned LLMs to infuse models with retrieval mechanisms to reduce hallucinations. |
| Outcome: | The proposed approach reduces the frequency of hallucinations by reducing the coverage of relevant facts and generating more informative responses while providing higher attribution rates. |
Unsupervised Labeled Parsing with Deep Inside-Outside Recursive Autoencoders (D19-1)
Copied to clipboard
| Challenge: | Existing models that use ground-truth part-of-speech tags are not always available and have significant weaknesses. |
| Approach: | They propose to use deep inside-outside recursive autoencoders to cluster the learned phrase vectors to induce span labels. |
| Outcome: | The proposed model outperforms ELMo and BERT on two versions of the Wall Street Journal dataset and improves over a previous state-of-the-art system that requires additional human annotations by 5 absolute F1 points (19% relative error reduction). |
Optimal Transport-based Alignment of Learned Character Representations for String Similarity (P19-1)
Copied to clipboard
| Challenge: | String similarity models are crucial for record linkage, data integration, search and entity resolution systems. |
| Approach: | They propose a model that encodes the characters of each string, aligns the encodings using Sinkhorn Iteration and scores the alignment with a convolutional neural network. |
| Outcome: | The proposed model outperforms state-of-the-art and classical similarity models on four of the five datasets and improves performance by applying it to cross-document coreference. |
Training Structured Prediction Energy Networks with Indirect Supervision (N18-2)
Copied to clipboard
| Challenge: | a new rank-based training method for structured prediction energy networks is proposed . structured prediction is important in many domains, including computer vision, computational biology and natural language processing. |
| Approach: | They propose a rank-based training method for structured prediction energy networks . they use a scoring function defined with domain knowledge to train the models . |
| Outcome: | The proposed method minimizes ranking violation of the sampled structures with respect to a scalar scoring function defined with domain knowledge. |
Marginal Likelihood Training of BiLSTM-CRF for Biomedical Named Entity Recognition from Disjoint Label Sets (D18-1)
Copied to clipboard
| Challenge: | Existing large labeled text datasets contain labels for multiple subsets of biomedical entity types, but it is rare to find large labeling datasets containing all desired entity types together. |
| Approach: | They propose a method for training a single CRF extractor from multiple datasets with disjoint or partially overlapping sets of entity types. |
| Outcome: | The proposed method improves NER F1 over training in isolation on biocreative V CDR, biocreativ VI ChemProt and MedMentions datasets. |
Diverse Distributions of Self-Supervised Tasks for Meta-Learning in NLP (2021.emnlp-main)
Copied to clipboard
| Challenge: | Meta-learning considers learning as an efficient learning process that can leverage its past experience to accurately solve new tasks. |
| Approach: | They propose to provide task distributions for meta-learning by considering self-supervised tasks automatically proposed from unlabeled text to enable large-scale meta- learning in NLP. |
| Outcome: | The proposed distributions show that human learning models perform better on the few-shot benchmark than previous methods. |
A Distant Supervision Corpus for Extracting Biomedical Relationships Between Chemicals, Diseases and Genes (2022.lrec-1)
Copied to clipboard
| Challenge: | Biomedical researchers have used manual curation to extract biomedical interactions from research texts to improve coverage. |
| Approach: | They propose a new dataset for training and evaluating multi-class multi-label biomedical relation extraction models using human annotations and the CTD database. |
| Outcome: | The proposed dataset is substantially larger and cleaner than existing datasets and includes annotations linking mentions to their entities. |
Efficient k-NN Search with Cross-Encoders using Adaptive Multi-Round CUR Decomposition (2023.findings-emnlp)
Copied to clipboard
| Challenge: | ANNCUR uses a cross-encoder only to perform k-NN search, but the approximation of the distances is often detrimental to the retrieval of top-k items. |
| Approach: | They propose a method that minimizes approximation error for k-nearest neighbor searches . they propose to use a cross-encoder only to perform k NN search . |
| Outcome: | The proposed method reduces approximation error for top-k neighbors by up to 70% . iteratively performs k-NN search using the available anchors, then adds them to the next set . |
Low-Resource Compositional Semantic Parsing with Concept Pretraining (2023.eacl-main)
Copied to clipboard
| Challenge: | Semantic parsing is a key role in voice assistants by mapping natural language to structured meaning representations. |
| Approach: | They propose an architecture to perform domain adaptation automatically with only a small amount of metadata about the new domain and without any new training data. |
| Outcome: | The proposed architecture outperforms existing models in low-resource settings. |
Multi-CLS BERT: An Efficient Alternative to Traditional Ensembling (2023.acl-long)
Copied to clipboard
| Challenge: | ensembling BERT models often improves accuracy but at the cost of significantly more computation and memory footprint. |
| Approach: | They propose a new ensembling method for CLS-based prediction tasks that is almost as efficient as a single BERT model. |
| Outcome: | The proposed method outperforms existing BERT models on GLUE and SuperGLUE with 100 training samples. |
Revisiting the Architectures like Pointer Networks to Efficiently Improve the Next Word Distribution, Summarization Factuality, and Beyond (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing solutions for word probability distributions are limited and the output softmax layer is inherently limited. |
| Approach: | They propose to use the output softmax layer to compute the word probability distribution instead of using pointer networks to break the bottleneck. |
| Outcome: | The proposed method improves factCC score by 2 points in CNN/DM and XSUM dataset, and MAUVE scores by 30% in bookSum paragraph-level dataset. |
OpenKI: Integrating Open Information Extraction and Knowledge Bases with Relation Inference (N19-1)
Copied to clipboard
| Challenge: | Existing methods for knowledge extraction and alignment are limited in quality and performance. |
| Approach: | They propose to integrate OpenIE extractions in the form of (subject, predicate, object) triples with Knowledge Bases (KB) |
| Outcome: | The proposed method improves state-of-the-art for OpenIE extractions and boosts performance on OpenIE from semi-structured data. |
DISAPERE: A Dataset for Discourse Structure in Peer Review Discussions (2022.naacl-main)
Copied to clipboard
Neha Kennard, Tim O’Gorman, Rajarshi Das, Akshay Sharma, Chhandak Bagchi, Matthew Clinton, Pranay Kumar Yelugam, Hamed Zamani, Andrew McCallum
| Challenge: | Prior work on labeling arguments extracted from peer review text has focused qualified labor force on labelling arguments extracted by the text. |
| Approach: | They synthesize label sets from prior work and extend them to include fine-grained annotations of review and rebuttal sentences. |
| Outcome: | The proposed dataset synthesizes label sets from prior work and extends them to include fine-grained annotation of review and rebuttal sentences. |
MS-Mentions: Consistently Annotating Entity Mentions in Materials Science Procedural Text (2021.emnlp-main)
Copied to clipboard
Tim O’Gorman, Zach Jensen, Sheshera Mysore, Kevin Huang, Rubayyat Mahbub, Elsa Olivetti, Andrew McCallum
| Challenge: | Material science synthesis procedures require high-quality annotations, which are limited by the size and quality of the annotations. |
| Approach: | They propose a corpus of entity mention annotations over 595 Material Science synthesis procedures. |
| Outcome: | The proposed approach greatly expands the training data available for the Named Entity Recognition task. |
Distributional Inclusion Vector Embedding for Unsupervised Hypernymy Detection (N18-1)
Copied to clipboard
| Challenge: | Existing unsupervised methods for learning hypernyms from unlabeled text are not scaled to large vocabularies or yield unacceptably poor accuracy. |
| Approach: | They propose an unsupervised method of hypernym discovery using word contexts . they use word2vec to embed word context distributions without supervision . |
| Outcome: | The proposed method provides double the precision and highest average performance on 11 datasets. |
Scaling Within Document Coreference to Long Texts (2021.findings-acl)
Copied to clipboard
Raghuveer Thirukovalluru, Nicholas Monath, Kumar Shridhar, Manzil Zaheer, Mrinmaya Sachan, Andrew McCallum
| Challenge: | Existing end-to-end coreference resolution models use expensive span representations and antecedent prediction mechanisms. |
| Approach: | They propose an approximation to end-to-end coreference resolution models which scales gracefully to documents of any length. |
| Outcome: | The proposed model reduces training and inference time and memory costs compared to current models with minimal loss in accuracy. |
Multi-step Entity-centric Information Retrieval for Multi-Hop Question Answering (D19-58)
Copied to clipboard
Rajarshi Das, Ameya Godbole, Dilip Kavarthapu, Zhiyu Gong, Abhishek Singhal, Mo Yu, Xiaoxiao Guo, Tian Gao, Hamed Zamani, Manzil Zaheer, Andrew McCallum
| Challenge: | Multi-hop question answering (QA) requires an information retrieval system that can find multiple supporting evidence needed to answer the question. |
| Approach: | They propose a technique that uses information of entities present in the initial retrieved evidence to learn to ‘hop’ onto other relevant evidence. |
| Outcome: | The proposed method boosts retrieval performance on a multi-hop question answering dataset with 5 million Wikipedia paragraphs and a model without training increases its performance by 10.59 F1. |
Long Document Summarization in a Low Resource Setting using Pretrained Language Models (2021.acl-srw)
Copied to clipboard
Ahsaas Bajaj, Pavitra Dangati, Kalpesh Krishna, Pradhiksha Ashok Kumar, Rheeya Uppaal, Bradford Windsor, Eliot Brenner, Dominic Dotterrer, Rajarshi Das, Andrew McCallum
| Challenge: | Existing abstractive summarization methods only achieve 17.9 ROUGE-L in low-resource settings. |
| Approach: | They propose to use a modern abstractive summarization algorithm to extract salient sentences from long documents to improve their performance. |
| Outcome: | The proposed method beats several competitive salience detection baselines and the identified salient sentences agree with independent human labeling by domain experts. |
Entity Linking via Explicit Mention-Mention Coreference Modeling (2022.naacl-main)
Copied to clipboard
| Challenge: | Using a learning approach for entity mentions is a key component of modern entity linking systems for both candidate generation and making linking predictions. |
| Approach: | They propose a training approach that builds minimum spanning arborescences over mentions and entities to explicitly model mention coreference relationships. |
| Outcome: | The proposed approach improves candidate generation recall and link accuracy on the biomedical dataset and on MedMentions, setting a new SOTA result in linking accuracy. |
Modeling Fine-Grained Entity Types with Box Embeddings (2021.acl-long)
Copied to clipboard
| Challenge: | Neural entity typing models typically represent fine-grained entity types as vectors in a high-dimensional space, but such spaces are not well-suited to modeling complex interdependencies. |
| Approach: | They propose to use box embeddings to embed types into a high-dimensional hyperrectangle space and then use it to hypothesize a type representation for the mention. |
| Outcome: | The proposed model captures latent type hierarchies better than a vector-based model on several entity typing benchmarks. |
Longtonotes: OntoNotes with Longer Coreference Chains (2023.findings-eacl)
Copied to clipboard
Kumar Shridhar, Nicholas Monath, Raghuveer Thirukovalluru, Alessandro Stolfo, Manzil Zaheer, Andrew McCallum, Mrinmaya Sachan
| Challenge: | Using Ontonotes, documents in certain genres were split into smaller parts for ease of annotation. |
| Approach: | They propose to merge annotations from documents split into smaller parts in Ontonotes for ease of annotation. |
| Outcome: | The proposed corpus restores documents to their original form, revealing dramatic increases in length in certain genres. |
Clustering-based Inference for Biomedical Entity Linking (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to linking entities ignore relationships between entities in biomedical knowledge bases. |
| Approach: | They propose a model which can link mentions of unseen entities using learned representations of entities. |
| Outcome: | The proposed model improves on the largest publicly available biomedical dataset by 3.0 points of accuracy and 2.3 points of reliability. |
Enhanced Distant Supervision with State-Change Information for Relation Extraction (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for enhancing distant supervision with state-change information for relation extraction are limited. |
| Approach: | They propose a method for enhancing distant supervision with state-change information for relation extraction by adding temporal information to a curation dataset. |
| Outcome: | The proposed method reduces noise when used for static relation extraction and can be used to train a relation-extraction system that detects a change of state in relations. |
Every Answer Matters: Evaluating Commonsense with Probabilistic Measures (2024.acl-long)
Copied to clipboard
Qi Cheng, Michael Boratko, Pranay Kumar Yelugam, Tim O’Gorman, Nalini Singh, Andrew McCallum, Xiang Li
| Challenge: | Existing commonsense evaluations are often posed as multiple-choice questions, allowing models to exploit systematic biases. |
| Approach: | They propose a generative task that evaluates common sense via multiple open-ended generations and a method that strongly correlates with human judgments. |
| Outcome: | The proposed method outperforms strong language model baselines on a dataset of human and machine common sense. |
xLM: A Python Package for Non-Autoregressive Language Models (2026.eacl-demo)
Copied to clipboard
| Challenge: | Autoregressive language models generate text sequentially from left to right by adding one token at a time. |
| Approach: | They propose a python package that provides a suite of small non-autoregressive language models that can be used by researchers. |
| Outcome: | The proposed package makes implementing small non-autoregressive language models faster and provides a suite of pre-trained models that can be used by the research community. |
An Interface for Annotating Science Questions (D18-2)
Copied to clipboard
Michael Boratko, Harshit Padigela, Divyendra Mikkilineni, Pritish Yuvraj, Rajarshi Das, Andrew McCallum, Maria Chang, Achille Fokoue, Pavan Kapanipathi, Nicholas Mattei, Ryan Musa, Kartik Talamadupula, Michael Witbrock
| Challenge: | a new interface for human annotation of science question-answer pairs with their knowledge and reasoning types is proposed . the interface is based on previous work on the ARC dataset, but does not provide clear definitions of these types of knowledge. |
| Approach: | They propose an interface for human annotation of science question-answer pairs with their respective knowledge and reasoning types. |
| Outcome: | The proposed interface improves the classification of science questions in a preliminary study involving 10 participants. |
Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction (N18-1)
Copied to clipboard
| Challenge: | Existing work in relation extraction ignores relationships expressed across sentence boundaries . document-level annotation is common in biological text . |
| Approach: | They propose a model which simultaneously predicts relationships between all mention pairs in a document. |
| Outcome: | The proposed model is larger than existing human-annotated biological information extraction datasets and more accurate than distantly supervised alternatives. |
Event-Event Relation Extraction using Probabilistic Box Embedding (2022.acl-short)
Copied to clipboard
| Challenge: | Existing frameworks of event relation extraction do not guarantee coherence between different relation types, such as anti-symmetry. |
| Approach: | They propose to modify existing ERE framework to guarantee coherence by representing each event as a box representation without applying explicit constraints. |
| Outcome: | The proposed model shows stronger conjunctive constraint satisfaction compared to previous models with constraint injection. |
Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoning (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to generalize knowledge bases model triple-level uncertainty . Existing models only model triple level uncertainty, and reasoning results lack global consistency. |
| Approach: | They propose a method to embed knowledge graphs with calibrated probabilistic semantics . they model each entity as a box and relations between two entities as affine transforms based on affinity transforms. |
| Outcome: | Experiments show that the proposed method outperforms baseline methods on confidence prediction and fact ranking. |
Word2Box: Capturing Set-Theoretic Semantics of Words using Box Embeddings (2022.acl-long)
Copied to clipboard
Shib Dasgupta, Michael Boratko, Siddhartha Mishra, Shriya Atmakuri, Dhruvesh Patel, Xiang Li, Andrew McCallum
| Challenge: | Word2Box provides a set-theoretic training objective for learning word representations . word representation is not natural, all senses and contexts, levels of abstraction, variants and modifications which the word may represent are forced to be captured by mat t is nunc. |
| Approach: | They propose a fuzzy-set interpretation of box embeddings and learn box representations of words using a set-theoretic training objective. |
| Outcome: | The proposed model improves word similarity tasks on less common words. |
Comparing Neighbors Together Makes it Easy: Jointly Comparing Multiple Candidates for Efficient and Effective Retrieval (2024.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that using only bi-encoders as an intermediate reranker can improve top-1 accuracy with negligible slowdown (7%). |
| Approach: | They propose a framework that compares a query and multiple embeddings of similar candidates through shallow self-attention layers, delivering rich representations contextualized to each other. |
| Outcome: | The proposed framework compares a query and multiple embeddings of similar candidates through shallow self-attention layers, delivering rich representations contextualized to each other. |
Case-based Reasoning for Natural Language Queries over Knowledge Bases (2021.emnlp-main)
Copied to clipboard
Rajarshi Das, Manzil Zaheer, Dung Thai, Ameya Godbole, Ethan Perez, Jay Yoon Lee, Lizhen Tan, Lazaros Polymenakos, Andrew McCallum
| Challenge: | Using human-labeled examples, case-based reasoning can solve complex problems from scratch . case-Based reasoning is a paradigm that is used to solve complex problem . |
| Approach: | They propose a neuro-symbolic CBR approach for question answering over large knowledge bases. |
| Outcome: | The proposed approach outperforms the current state of the art on a CWQ dataset by 11% on accuracy. |
An Instance Level Approach for Shallow Semantic Parsing in Scientific Procedural Text (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to parse scientific text using grammatically similar labeled sentences are limited and expensive to create. |
| Approach: | They propose a method where semantic labels from structurally similar sentences are copied to test sentences. |
| Outcome: | The proposed approach outperforms baseline and prior methods by 0.75 to 3 F1 absolute in the wet lab protocol corpus and 1 F1 absolut in the materials science procedural text corpus. |
Learning to Few-Shot Learn Across Diverse Natural Language Classification Tasks (2020.coling-main)
Copied to clipboard
| Challenge: | Pre-trained transformer models have shown great success in improving performance on downstream tasks, but fine-tuning on a new task still requires large amounts of labeled data. |
| Approach: | They propose a method which allows optimization-based meta-learning across tasks . they use transformers to train transformer models and find better generalizations . |
| Outcome: | The proposed method outperforms self-supervised training and pre-trained models on 17 NLP tasks. |
Box Embeddings: An open-source library for representation learning using geometric structures (2021.emnlp-demo)
Copied to clipboard
Tejas Chheda, Purujit Goyal, Trang Tran, Dhruvesh Patel, Michael Boratko, Shib Sankar Dasgupta, Andrew McCallum
| Challenge: | Recent studies have explored alternative vector representations with different inductive biases or capabilities. |
| Approach: | They propose a Python library that extends probabilistic box embeddings to geometric shapes and regions. |
| Outcome: | The proposed library is fully open source and compatible with PyTorch and TensorFlow. |
Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Auto-Encoders (N19-1)
Copied to clipboard
| Challenge: | Using the deep inside-outside recursive autoencoder, we can extract both shallow parses and full syntactic trees from any domain or language automatically. |
| Approach: | They propose a fully-unsupervised method for discovering syntax that simultaneously learns representations for constituents within the induced tree. |
| Outcome: | The proposed method outperforms previous methods on the WSJ dataset. |
ProtoQA: A Question Answering Dataset for Prototypical Common-Sense Reasoning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing question answering datasets for common sense reasoning are lacking for prototypical situations. |
| Approach: | They propose a question answering dataset for training and evaluating common sense reasoning capabilities of artificial intelligence systems in such prototypical situations. |
| Outcome: | The proposed model outperforms existing models on all evaluation metrics with a meaningful gap. |
Energy and Policy Considerations for Deep Learning in NLP (P19-1)
Copied to clipboard
| Challenge: | Recent advances in hardware and methodology for training neural networks have enabled significant accuracy improvements across many NLP tasks. |
| Approach: | They quantify the approximate financial and environmental costs of training neural network models . they propose actionable recommendations to reduce costs and improve equity in NLP research . |
| Outcome: | The proposed recommendations address the cost and environmental costs of training neural networks for NLP. |
Machine Reading Comprehension using Case-based Reasoning (2023.findings-emnlp)
Copied to clipboard
Dung Thai, Dhruv Agarwal, Mudit Chaudhary, Wenlong Zhao, Rajarshi Das, Jay-Yoon Lee, Hannaneh Hajishirzi, Manzil Zaheer, Andrew McCallum
| Challenge: | Current state-of-the-art machine readers do not support case-based reasoning . |
| Approach: | They propose a method that extracts a set of similar cases from a nonparametric memory and then predicts an answer by selecting the span in the test context that is most similar to the contextualized representations of answers. |
| Outcome: | The proposed method outperforms baselines on NaturalQuestions and NewsQA by 11.5 and 8.4 EM. |
Efficient Nearest Neighbor Search for Cross-Encoder Models using Matrix Factorization (2022.emnlp-main)
Copied to clipboard
| Challenge: | Efficient k-nearest neighbor search is a fundamental task, foundational for many problems in NLP. |
| Approach: | They propose an approach that avoids the use of a dual-encoder for retrieval, relying solely on the cross-encoding model. |
| Outcome: | Empirically, for k > 10, our approach provides test-time recall-vs-computational cost trade-offs superior to the current widely-used methods that re-rank items retrieved using a dual-encoder or TF-IDF. |
Multistage Collaborative Knowledge Distillation from a Large Language Model for Semi-Supervised Sequence Generation (2024.acl-long)
Copied to clipboard
Jiachen Zhao, Wenlong Zhao, Andrew Drozdov, Benjamin Rozonoyer, Md Arafat Sultan, Jay-Yoon Lee, Mohit Iyyer, Andrew McCallum
| Challenge: | Low-resource tasks such as semi-supervised sequence generation require expert knowledge and cost. |
| Approach: | They propose a method for semi-supervised sequence generation where few examples are too scarce to fine tune a model. |
| Outcome: | The proposed method can generalize better than its teacher to unseen examples on semi-supervised sequence generation tasks. |
Unsupervised Parsing with S-DIORA: Single Tree Encoding for Deep Inside-Outside Recursive Autoencoders (2020.emnlp-main)
Copied to clipboard
| Challenge: | Syntactic parse trees are valuable intermediate features for many NLP pipelines. |
| Approach: | They propose an improved version of DIORA that encodes a single tree rather than a softly-weighted mixture of trees by employing a hard argmax operation and a beam at each cell in the chart. |
| Outcome: | The proposed model improves state-of-the-art in constituency parsing on the English WSJ Penn Treebank by 2.2-6% F1, depending on the data used for fine-tuning. |
Probabilistic Case-based Reasoning for Open-World Knowledge Graph Completion (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for learning non-parametric representations of entities and relations are based on tensor factorization or sophisticated neural approaches. |
| Approach: | They propose a case-based reasoning system that retrieves ‘cases’ that are similar to the given problem and then stores them in its parameters. |
| Outcome: | The proposed model outperforms state-of-the-art methods on several benchmark datasets and is non-parametric and grows dynamically as new entities and relations arrive in the KB. |
Chains-of-Reasoning at TextGraphs 2019 Shared Task: Reasoning over Chains of Facts for Explainable Multi-hop Inference (D19-53)
Copied to clipboard
| Challenge: | EMNLP 2019 shared task on 'Multi-hop Inference Explanation Regeneration' identifies chains of facts relevant to explain an answer to an elementary science examination question. |
| Approach: | They propose a system that identifies chains of facts relevant to explain an answer to an elementary science examination question. |
| Outcome: | The proposed system outperforms the second best system by 14.95 points on the mean average precision (MAP) metric. |
Energy-Based Reranking: Improving Neural Machine Translation Using Energy-Based Models (2021.acl-long)
Copied to clipboard
Sumanta Bhattacharyya, Amirmohammad Rooshenas, Subhajit Naskar, Simeng Sun, Mohit Iyyer, Andrew McCallum
| Challenge: | Autoregressive neural machine translation (NMT) uses a tractable likelihood computation and efficient sampling. |
| Approach: | They propose to use an energy-based model to mimic the behavior of the task measure and use it to train an energy based re-ranking algorithm. |
| Outcome: | The proposed model improves on the samples drawn from the NMT with a higher BLEU score than the experimental model and the energy-based re-ranking algorithm. |
PaRaDe: Passage Ranking using Demonstrations with LLMs (2023.findings-emnlp)
Copied to clipboard
Andrew Drozdov, Honglei Zhuang, Zhuyun Dai, Zhen Qin, Razieh Rahimi, Xuanhui Wang, Dana Alon, Mohit Iyyer, Andrew McCallum, Donald Metzler, Kai Hui
| Challenge: | Existing studies show that large language models can be instructed to perform zero-shot passage re-ranking . Existing work like UPR demonstrate promising results for zero- shot ranking using LLMs . |
| Approach: | They propose a demonstration selection strategy based on difficulty rather than semantic similarity . they propose to include only one demonstration in the prompt to improve re-ranking . |
| Outcome: | The proposed method improves LLM-based re-ranking by adding one demonstration to the prompt. |
A2N: Attending to Neighbors for Knowledge Graph Inference (P19-1)
Copied to clipboard
| Challenge: | Existing knowledge graph completion methods learn a fixed embedding for every entity, which is suboptimal as it requires memorizing and generalizing to all possible entity relationships. |
| Approach: | They propose a method which learns query-dependent representations of entities by combining relevant neighborhood of an entity. |
| Outcome: | The proposed model performs competitively or better than existing state-of-the-art models for knowledge graph completion. |
Linguistically-Informed Self-Attention for Semantic Role Labeling (D18-1)
Copied to clipboard
| Challenge: | Existing models of semantic role labeling use no explicit linguistic features. prior work has shown that syntax trees can dramatically improve SRL decoding. |
| Approach: | They propose a neural network model that incorporates syntax using only raw tokens . they show that LISA out-performs the state-of-the-art with contextually-encoded word representations a 1.0 F1 on newswire and 2.0 F1 in out-of domain text . |
| Outcome: | The proposed model outperforms the state-of-the-art model with word embeddings and predicted predicates. |
Causal Matching with Text Embeddings: A Case Study in Estimating the Causal Effects of Peer Review Policies (2023.findings-acl)
Copied to clipboard
| Challenge: | Using a method that uses text embeddings, we find that human judges prefer manuscript matches from our method in 70% of cases. |
| Approach: | They propose to use text embeddings to integrate a method to estimate the causal effects of peer review policies in publication venues that shift policies from single-blind to double-blinded. |
| Outcome: | The proposed method is compared with a widely-used method of stratified propensity score matching and a baseline of randomly selected matches. |
Hierarchical Losses and New Resources for Fine-grained Entity Typing and Linking (P18-1)
Copied to clipboard
| Challenge: | Existing methods to incorporate hierarchical information into knowledge bases have yielded little benefit. |
| Approach: | They propose methods to integrate hierarchical information using real bilinear mappings . they also propose two new datasets containing wide and deep hierarchies . |
| Outcome: | The proposed methods improve on flat predictions and fine-grained entity typing on FIGER dataset. |
Multi-facet Universal Schema (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods to solve relation extraction tasks violate USchema's assumption that sentence patterns that share the same entity pairs are similar to each other. |
| Approach: | They propose a multi-facet universal schema that embeds multiple sentence patterns as facets and encourages one to be close to that of another if they co-occur with the same entity pair. |
| Outcome: | The proposed model outperforms its single-facet embedding counterpart in relation extraction tasks. |
Improved Latent Tree Induction with Distant Supervision via Span Constraints (2021.emnlp-main)
Copied to clipboard
Zhiyang Xu, Andrew Drozdov, Jay Yoon Lee, Tim O’Gorman, Subendhu Rongali, Dylan Finkbeiner, Shilpa Suresh, Mohit Iyyer, Andrew McCallum
| Challenge: | Distant supervision is not a practical way to perform unsupervised syntactic parsing. |
| Approach: | They propose a technique that uses distant supervision to improve unsupervised constituency parsing by using phrase bracketing. |
| Outcome: | The proposed method improves constituency parsing on English WSJ Penn Treebank by more than 5 F1 compared with full parse tree annotations. |
Benchmarking Scalable Methods for Streaming Cross Document Entity Coreference (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to disambiguate mentions of named entities are limited . existing approaches omit details needed to ensure fair comparisons . |
| Approach: | They propose to use streaming CDC to disambiguate mentions of named entities . they compare a set of existing and new datasets to evaluate their models . |
| Outcome: | The proposed system is well-suited for processing streams of data where new entities are frequently introduced. |
You can’t pick your neighbors, or can you? When and How to Rely on Retrieval in the kNN-LM (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that retrieval-enhanced language models can improve perplexity by combining text from large external datastores with a k-nearest neighbors model. |
| Approach: | They propose a retrieval-enhanced language model that interpolates existing LMs with a k-nearest neighbors model and requires no additional training. |
| Outcome: | The proposed model improves on two English language modeling datasets and shows that it is most effective when items have high semantic similarity with the query. |
Inducing and Using Alignments for Transition-based AMR Parsing (2022.naacl-main)
Copied to clipboard
Andrew Drozdov, Jiawei Zhou, Radu Florian, Andrew McCallum, Tahira Naseem, Yoon Kim, Ramón Astudillo
| Challenge: | Abstract Meaning Representation parsers rely on node-to-word alignments, but lack the complexity of the pipeline. |
| Approach: | They propose a neural aligner for abstract meaning representation that learns node-to-word alignments without relying on pipelines. |
| Outcome: | The proposed approach improves accuracy and generalization from AMR2.0 to AMR3.0 corpora. |
Changing the Mind of Transformers for Topically-Controllable Language Generation (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing interactive writing assistants do not allow authors to guide text generation in desired topical directions. |
| Approach: | They propose a framework that displays multiple candidate upcoming topics and generates a text generation model that adheres to the chosen topics. |
| Outcome: | The proposed model generates fluent sentences related to the selected topics, as judged by automated metrics and crowdsourced workers. |
Softmax Bottleneck Makes Language Models Unable to Represent Multi-mode Word Distributions (2022.acl-long)
Copied to clipboard
| Challenge: | Existing language models can't predict the probability distribution over next words or masked words because of a single hidden state. |
| Approach: | They propose a multi-facet softmax to address the limitations of softmax . they propose LMs that can predict the next word distribution better as their sizes grow . |
| Outcome: | The proposed model achieves two-fold improvements in the perplexity of GPT-2 and BERT. |