Papers by Jacob Eisenstein
Sparse, Dense, and Attentional Representations for Text Retrieval (2021.tacl-1)
Copied to clipboard
| Challenge: | Dual encoders perform retrieval by encoding documents and queries into dense low-dimensional vectors, scoring each document by its inner product with the query. |
| Approach: | They propose a dual-encoder-based neural model that combines the efficiency of dual encoders with expressiveness of more costly attentional architectures. |
| Outcome: | The proposed model outperforms strong alternatives in large-scale retrieval. |
Explainable Prediction of Medical Codes from Clinical Text (N18-1)
Copied to clipboard
| Challenge: | Clinical notes are text documents that are created by clinicians for each patient encounter. |
| Approach: | They propose a method that aggregates information across the document using a convolutional neural network and uses an attention mechanism to select the most relevant segments for each of the thousands of possible codes. |
| Outcome: | The proposed method is accurate and better than the current state of the art. |
Si O No, Que Penses? Catalonian Independence and Linguistic Identity on Social Media (N18-2)
Copied to clipboard
| Challenge: | This study examines the use of Catalan on Twitter in discourse related to the 2017 independence referendum. |
| Approach: | They use code-switching to determine the role of Catalan in political discourse . they corroborate prior findings that pro-independence tweets are more likely to include the local language than anti-independent tweets . |
| Outcome: | The proposed method corroborates previous findings that pro-independence tweets are more likely to include the local language than anti-independent tweets. |
Predicting Semantic Relations using Global Graph Properties (D18-1)
Copied to clipboard
| Challenge: | Semantic graphs encode the structural qualities of language as a representation of human knowledge. |
| Approach: | They propose a global-theoretic model that integrates global and local properties of semantic graphs to improve local prediction of relational relations between synsets. |
| Outcome: | The proposed model improves on the local task of predicting semantic relations between synsets, yielding state-of-the-art results on the WN18RR dataset. |
Predicting Long-Term Citations from Short-Term Linguistic Influence (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to quantify linguistic influence in timestamped documents are not informative about extent to which a paper affected subsequent publications. |
| Approach: | They propose to quantify linguistic influence in timestamped document collections by estimating a Hawkes process with a low-rank parameter matrix and identify lexical and semantic changes using contextual embeddings and word frequencies. |
| Outcome: | The proposed method is based on an online evaluation with incremental temporal training/test splits, in comparison with a strong baseline that includes predictors for initial citation counts, topics, and lexical features. |
Learning to Recognize Dialect Features (2021.naacl-main)
Copied to clipboard
| Challenge: | linguistics do not characterize dialects as simple categories, but as collections of correlated features. |
| Approach: | They propose two multitask learning approaches based on pretrained transformers to detect dialect features in speech and text. |
| Outcome: | The proposed models learn to recognize many features with high accuracy on 22 dialect features of Indian English. |
Informativeness and Invariance: Two Perspectives on Spurious Correlations in Natural Language (2022.naacl-main)
Copied to clipboard
| Challenge: | Spurious correlations are a threat to the trustworthiness of natural language processing systems. |
| Approach: | They propose a definition of spurious correlations in terms of conditional probabilities and a generalized definition of the term . they propose UIs that allow individual input features to be independent of labels. |
| Outcome: | The proposed definition can be generalized from uniformity to independence without affecting the claims of the paper. |
Will it Unblend? (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Blends, such as “innoventor”, are one particularly challenging class of OOV terms, as they are formed by fusing together two or more bases that relate to the intended meaning in unpredictable manners and degrees. |
| Approach: | They propose to use a dataset of English OOV blends to quantify the difficulty of interpreting the meanings of blends by large-scale contextual language models such as BERT. |
| Outcome: | The proposed model outperforms character-level and context-free embeddings, although their results are still far from satisfactory. |
Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond (2022.tacl-1)
Copied to clipboard
Amir Feder, Katherine A. Keith, Emaad Manzoor, Reid Pryzant, Dhanya Sridhar, Zach Wood-Doughty, Jacob Eisenstein, Justin Grimmer, Roi Reichart, Margaret E. Roberts, Brandon M. Stewart, Victor Veitch, Diyi Yang
| Challenge: | causality has not had the same importance in natural language processing, says aaron e. smith . he says research on causality in NLP remains scattered across domains without unified definitions . |
| Approach: | They propose to consolidate research on causality in NLP across academic areas . they explore potential uses of causal inference to improve robustness, fairness, interpretability . |
| Outcome: | The proposed method is a unified overview of causal inference for the NLP community. |
The Referential Reader: A Recurrent Entity Network for Anaphora Resolution (P19-1)
Copied to clipboard
| Challenge: | Existing methods for storing and accessing entity mentions are expensive and implausible for human readers. |
| Approach: | They propose a method for storing and accessing entity mentions during online text processing. |
| Outcome: | The proposed model performs well on a dataset of pronoun-name anaphora. |
Time-Aware Language Models as Temporal Knowledge Bases (2022.tacl-1)
Copied to clipboard
Bhuwan Dhingra, Jeremy R. Cole, Julian Martin Eisenschlos, Daniel Gillick, Jacob Eisenstein, William W. Cohen
| Challenge: | Existing language models are trained on snapshots of data collected at a specific moment in time. |
| Approach: | They propose a diagnostic dataset aimed at probing LMs for factual knowledge that changes over time. |
| Outcome: | The proposed method improves memorization of seen facts and calibration on unseen facts from future time periods. |
Reuse Your Rewards: Reward Model Transfer for Zero-Shot Cross-Lingual Alignment (2024.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual human preference data are difficult to obtain at scale, making it challenging to extend this framework to diverse languages. |
| Approach: | They propose a method where a reward model is trained on preference data in one source language and applied to other target languages. |
| Outcome: | The proposed approach is effective under comprehensive evaluation settings, including human evaluation. |
AdvAug: Robust Adversarial Augmentation for Neural Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work in neural machine translation has led to dramatic improvements in both research and commercial systems. |
| Approach: | They propose a adversarial augmentation method for Neural Machine Translation that minimizes vicinal risk over virtual sentences . they use a novel vicinity distribution for adversarials to describe a smooth interpolated embedding space . |
| Outcome: | The proposed method outperforms the current method on Chinese-English, English-French, and English-German translation benchmarks. |
Making “fetch” happen: The influence of social and linguistic context on nonstandard word growth and decline (D18-1)
Copied to clipboard
| Challenge: | In an online community, new words come and go, but language change is shaped and constrained by the grammatical system in which it takes part. |
| Approach: | They analysed the frequency of non-standard words in reddit to determine their impact on language change. |
| Outcome: | The results show that language change is shaped and constrained by the grammatical system in which it takes place. |
Measuring and Modeling Language Change (N19-5)
Copied to clipboard
| Challenge: | This tutorial will help researchers answer questions fundamental to the social sciences and humanities . |
| Approach: | This tutorial is designed to help researchers answer questions in the social sciences and humanities . it synthesizes recent computational techniques for handling and modeling temporal data . |
| Outcome: | The tutorial will synthesize recent techniques for handling and modeling temporal data, such as dynamic word embeddings, and identify useful tools for social scientists and digital humanities scholars. |
Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translation (D19-55)
Copied to clipboard
| Challenge: | Recent machine translation methods are highly sensitive to orthographical variations such as spelling errors. |
| Approach: | They propose to train machine translation models with random synthetic noise at training time . they focus on translation performance on natural typos, and show robustness to such noise . |
| Outcome: | The proposed method significantly improves translation models on natural typos without accessing natural noise data or distribution. |
Selectively Answering Ambiguous Questions (2023.emnlp-main)
Copied to clipboard
| Challenge: | Prior work has focused on the case in which the question is clear and the answer is unambiguous but possibly unknown. |
| Approach: | They propose to use a sampled set of questions to calibrate answers to ambiguous questions with varying model scales. |
| Outcome: | The results show that sampling-based confidence scores help calibrate answers to relatively unambiguous questions, with more dramatic improvements on ambiguous ones. |
Unsupervised Domain Adaptation of Contextualized Embeddings for Sequence Labeling (D19-1)
Copied to clipboard
| Challenge: | Contextualized word embeddings are becoming a ubiquitous component of natural language processing. |
| Approach: | They propose a domain-adaptive fine-tuning approach to pretrain on unlabeled text . they test this approach on sequence labeling in two challenging domains . |
| Outcome: | The proposed approach improves on sequence labeling in two domains: Early Modern English and Twitter. |
Predicting the Target Word of Game-playing Conversations using a Low-Rank Dialect Adapter for Decoder Models (2025.naacl-short)
Copied to clipboard
| Challenge: | Existing work proposes dialect adaptation for encoder models or encoder-decoder models. |
| Approach: | They propose to use MD-3 to combine task adapters and dialect adapters to decoder models using a masked word game-playing conversation. |
| Outcome: | The proposed architecture outperforms baselines on Indian English and Nigerian English on a masked conversation with two models. |
Dialect-robust Evaluation of Generated Text (2023.acl-long)
Copied to clipboard
Jiao Sun, Thibault Sellam, Elizabeth Clark, Tu Vu, Timothy Dozat, Dan Garrette, Aditya Siddhant, Jacob Eisenstein, Sebastian Gehrmann
| Challenge: | Existing evaluation metrics that are not robust to dialect variation are difficult to measure for many groups of users and can penalize systems for producing text in lower-resource dialects. |
| Approach: | They propose a dialect-robust evaluation metric that produces the same score for system outputs that share the same semantics but are expressed in different dialects. |
| Outcome: | The proposed method significantly improves dialect robustness while preserving the correlation between automated metrics and human ratings. |