Papers by Karl Stratos
Mining Knowledge for Natural Language Inference from Wikipedia Categories (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Accurate lexical entailment (LE) and natural language inference (NLI) tasks require expensive annotations. |
| Approach: | They propose to pretrain Wikipedia categories for lexical entailment and natural language inference by pretraining them on WikiNLI and transferring them to other knowledge bases. |
| Outcome: | The proposed model can improve strong baselines such as BERT and RoBERTa by pretraining on WikiNLI and transferring the models on downstream tasks. |
Mutual Information Maximization for Simple and Accurate Part-Of-Speech Induction (N19-1)
Copied to clipboard
| Challenge: | Using stochastic gradient descent, part-of-speech (POS) induction is a challenging task. |
| Approach: | They propose to maximize mutual information between the induced label and its context by maximizing mutual information. |
| Outcome: | The proposed approach achieves strong performance on a multitude of datasets and languages with a simple architecture that encodes morphology and context. |
Understanding Hard Negatives in Noise Contrastive Estimation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing theoretical results in contrastive learning focus on unconditional negative distributions. |
| Approach: | They propose a method where highest-scoring incorrect labels are chosen as negatives . they also propose 'hard negative mining' where negatives are used as negative examples . |
| Outcome: | The proposed approach achieves strong results on the task of zero-shot entity linking. |
Data-to-text Generation by Splicing Together Nearest Neighbors (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on data-to-text generation relies on retrieved "neighbors" but instead generates text token-by-token, left-to right. |
| Approach: | They propose to splice together retrieved segments of text from "neighbor" source-target pairs to generate text token-by-token, left-to-right. |
| Outcome: | The proposed method performs on par with strong baselines in terms of automatic and human evaluation, but allows for more interpretable and controllable generation. |
Seq2seq is All You Need for Coreference Resolution (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on coreference resolution suggests task-specific models are necessary . a recent line of work that take an alternative approach leveraging advances in seq2seq-based models is needed . |
| Approach: | They propose a pretrained seq2seq transformer to map an input document to a tagged sequence encoding the coreference annotation. |
| Outcome: | The proposed model outperforms or matches the best coreference systems on an array of datasets. |
Unsupervised Label Refinement Improves Dataless Text Classification (2021.findings-acl)
Copied to clipboard
| Challenge: | Prior work has shown that dataless classifiers can classify text into categories without using annotated training data. |
| Approach: | They propose a clustering-based approach that assigns a score to any document paired with a label description. |
| Outcome: | The proposed method improves classification performance across multiple datasets and makes it more robust to label descriptions. |
Improving Multitask Retrieval by Promoting Task Specialization (2023.tacl-1)
Copied to clipboard
| Challenge: | despite its practical appeal, naive multitask retrieval lags behind task-specific retrieval. |
| Approach: | They propose to train a multitask retriever that promotes task specialization . the model is highly performant on the KILT benchmark . |
| Outcome: | The proposed model outperforms task-specific retrievals on the KILT benchmark . it learns parameters that are more task-specialized than naive retrieval without prompting or adaptive learning. |
ImpRAG: Retrieval-Augmented Generation with Implicit Queries (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) systems treat retrieval and generation as separate processes, requiring explicit textual queries to connect them. |
| Approach: | They propose a query-free RAG system that integrates retrieval and generation into a unified model. |
| Outcome: | The proposed system can achieve 3.6-11.5 accuracy improvements on unseen tasks . it allows models to express their information needs without human-specified queries . |
The Impact of Visual Information in Chinese Characters: Evaluating Large Models’ Ability to Recognize and Utilize Radicals (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing studies have examined whether large language models and vision-language models can harness these sub-character features in Chinese through prompting. |
| Approach: | They establish a benchmark to evaluate large language models' understanding of visual elements in Chinese characters, including radicals, composition structures, strokes, and stroke counts. |
| Outcome: | The proposed model exhibits some, but still limited, knowledge of the visual elements in Chinese characters regardless of whether images of characters are provided. |
Model Editing by Standard Fine-Tuning (2024.findings-acl)
Copied to clipboard
| Challenge: | specialized methods for model editing are not as effective due to poor performance . standard fine-tuning alone can yield competitive model editing performance if it is modified . |
| Approach: | They propose to optimize conditional likelihood rather than the full likelihood . they also train on random or similar unedited facts to encourage locality . |
| Outcome: | The proposed model editing method outperforms specialized models in terms of edit score. |
Discrete Latent Variable Representations for Low-Resource Text Classification (2020.acl-main)
Copied to clipboard
| Challenge: | Several approaches to learning discrete latent variable models for text are available. |
| Approach: | They compare several approaches to learning discrete latent variable models for text in the case where exact marginalization over these variables is intractable. |
| Outcome: | The learned models outperform the previous best models in low-resource settings while learning significantly more compressed representations. |
EntEval: A Holistic Evaluation Benchmark for Entity Representations (D19-1)
Copied to clipboard
| Challenge: | EntEval is a test suite of tasks that require nontrivial understanding of entities. |
| Approach: | They propose to encode the mention context or the Wikipedia hyperlink annotations to learn better entity representations. |
| Outcome: | The proposed model improves strong baselines on multiple EntEval tasks. |
Label-Agnostic Sequence Labeling by Copying Nearest Neighbors (P19-1)
Copied to clipboard
| Challenge: | Retrieve-and-edit based structured prediction models condition on retrieved neighbors to generate new structures, but many models do not explicitly capture the discrete operations that allow for the neighbors to be edited into the target structure. |
| Approach: | They propose to explicitly condition on retrieved neighbors to create new structures . they propose to use a dynamic programming approach to sequence labeling . |
| Outcome: | The proposed model can perform accurate sequence labeling by explicitly copying labels from retrieved neighbors. |