Papers by Andrew Drozdov
Unsupervised Labeled Parsing with Deep Inside-Outside Recursive Autoencoders (D19-1)
Copied to clipboard
| Challenge: | Existing models that use ground-truth part-of-speech tags are not always available and have significant weaknesses. |
| Approach: | They propose to use deep inside-outside recursive autoencoders to cluster the learned phrase vectors to induce span labels. |
| Outcome: | The proposed model outperforms ELMo and BERT on two versions of the Wall Street Journal dataset and improves over a previous state-of-the-art system that requires additional human annotations by 5 absolute F1 points (19% relative error reduction). |
Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Auto-Encoders (N19-1)
Copied to clipboard
| Challenge: | Using the deep inside-outside recursive autoencoder, we can extract both shallow parses and full syntactic trees from any domain or language automatically. |
| Approach: | They propose a fully-unsupervised method for discovering syntax that simultaneously learns representations for constituents within the induced tree. |
| Outcome: | The proposed method outperforms previous methods on the WSJ dataset. |
kNN-LM Does Not Improve Open-ended Text Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Interpolation-based retrieval-augmented language models (LMs) are a subtype of retrieval augmented language model that computes the probability of the next token by interpolating between the softmax distribution of the original LM and a token distribution formed by retrieving over an external datastore. |
| Approach: | They propose to interpolate the predicted distribution of the next word with a distribution formed from the most relevant retrievals for a given prefix. |
| Outcome: | The proposed methods do not exhibit improvements in open-ended generation quality, as measured by automatic evaluation metrics and human evaluations. |
Multistage Collaborative Knowledge Distillation from a Large Language Model for Semi-Supervised Sequence Generation (2024.acl-long)
Copied to clipboard
Jiachen Zhao, Wenlong Zhao, Andrew Drozdov, Benjamin Rozonoyer, Md Arafat Sultan, Jay-Yoon Lee, Mohit Iyyer, Andrew McCallum
| Challenge: | Low-resource tasks such as semi-supervised sequence generation require expert knowledge and cost. |
| Approach: | They propose a method for semi-supervised sequence generation where few examples are too scarce to fine tune a model. |
| Outcome: | The proposed method can generalize better than its teacher to unseen examples on semi-supervised sequence generation tasks. |
Unsupervised Parsing with S-DIORA: Single Tree Encoding for Deep Inside-Outside Recursive Autoencoders (2020.emnlp-main)
Copied to clipboard
| Challenge: | Syntactic parse trees are valuable intermediate features for many NLP pipelines. |
| Approach: | They propose an improved version of DIORA that encodes a single tree rather than a softly-weighted mixture of trees by employing a hard argmax operation and a beam at each cell in the chart. |
| Outcome: | The proposed model improves state-of-the-art in constituency parsing on the English WSJ Penn Treebank by 2.2-6% F1, depending on the data used for fine-tuning. |
PaRaDe: Passage Ranking using Demonstrations with LLMs (2023.findings-emnlp)
Copied to clipboard
Andrew Drozdov, Honglei Zhuang, Zhuyun Dai, Zhen Qin, Razieh Rahimi, Xuanhui Wang, Dana Alon, Mohit Iyyer, Andrew McCallum, Donald Metzler, Kai Hui
| Challenge: | Existing studies show that large language models can be instructed to perform zero-shot passage re-ranking . Existing work like UPR demonstrate promising results for zero- shot ranking using LLMs . |
| Approach: | They propose a demonstration selection strategy based on difficulty rather than semantic similarity . they propose to include only one demonstration in the prompt to improve re-ranking . |
| Outcome: | The proposed method improves LLM-based re-ranking by adding one demonstration to the prompt. |
Improved Latent Tree Induction with Distant Supervision via Span Constraints (2021.emnlp-main)
Copied to clipboard
Zhiyang Xu, Andrew Drozdov, Jay Yoon Lee, Tim O’Gorman, Subendhu Rongali, Dylan Finkbeiner, Shilpa Suresh, Mohit Iyyer, Andrew McCallum
| Challenge: | Distant supervision is not a practical way to perform unsupervised syntactic parsing. |
| Approach: | They propose a technique that uses distant supervision to improve unsupervised constituency parsing by using phrase bracketing. |
| Outcome: | The proposed method improves constituency parsing on English WSJ Penn Treebank by more than 5 F1 compared with full parse tree annotations. |
You can’t pick your neighbors, or can you? When and How to Rely on Retrieval in the kNN-LM (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that retrieval-enhanced language models can improve perplexity by combining text from large external datastores with a k-nearest neighbors model. |
| Approach: | They propose a retrieval-enhanced language model that interpolates existing LMs with a k-nearest neighbors model and requires no additional training. |
| Outcome: | The proposed model improves on two English language modeling datasets and shows that it is most effective when items have high semantic similarity with the query. |
Inducing and Using Alignments for Transition-based AMR Parsing (2022.naacl-main)
Copied to clipboard
Andrew Drozdov, Jiawei Zhou, Radu Florian, Andrew McCallum, Tahira Naseem, Yoon Kim, Ramón Astudillo
| Challenge: | Abstract Meaning Representation parsers rely on node-to-word alignments, but lack the complexity of the pipeline. |
| Approach: | They propose a neural aligner for abstract meaning representation that learns node-to-word alignments without relying on pipelines. |
| Outcome: | The proposed approach improves accuracy and generalization from AMR2.0 to AMR3.0 corpora. |