Papers by Tom Kwiatkowski
MOLEMAN: Mention-Only Linking of Entities with a Mention Annotation Network (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to entity linking represent each entity with a single vector, but instead use a contextualized mention-encoder that learns to place similar mentions of the same entity closer in vector space than mentions from different entities. |
| Approach: | They propose an instance-based nearest neighbor approach to entity linking that allows for a contextualized mention-encoder to learn to place similar mentions of the same entity closer in vector space than mentions from different entities. |
| Outcome: | The proposed approach outperforms all other systems on two multilingual benchmarks and is simpler to train and interpretable. |
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions (N19-1)
Copied to clipboard
| Challenge: | In this paper we build a reading comprehension dataset of yes/no questions that are naturally occurring . they often query for complex, non-factoid information, and require difficult entailment-like inference to solve. |
| Approach: | They build a reading comprehension dataset of yes/no questions that are naturally occurring . they find they are unexpectedly challenging and require difficult inferences to solve . |
| Outcome: | The proposed method achieves 80.4% accuracy compared to 90% accuracy of human annotators and 62% majority-baseline. |
Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index (P19-1)
Copied to clipboard
| Challenge: | Existing open-domain question answering models require multiple documents on-demand for every input query. |
| Approach: | They propose query-agnostic indexable representations of document phrases that can drastically speed up open-domain question answering. |
| Outcome: | The proposed model can be trained and deployed even in a single 4-GPU server. |
Phrase-Indexed Question Answering: A New Challenge for Scalable Document Comprehension (D18-1)
Copied to clipboard
| Challenge: | Existing QA models rely on learning interaction between document and question . current models require explicit attention to the document before or as it reads it . |
| Approach: | They propose a modular question answering task that enforces complete independence of the document encoder from the question encoder. |
| Outcome: | The proposed model achieves reasonable accuracy but significantly underperforms unconstrained QA models. |
Evaluating and Modeling Attribution for Cross-Lingual Question Answering (2023.emnlp-main)
Copied to clipboard
Benjamin Muller, John Wieting, Jonathan Clark, Tom Kwiatkowski, Sebastian Ruder, Livio Soares, Roee Aharoni, Jonathan Herzig, Xinyi Wang
| Challenge: | Open-retrieval question answering systems are lacking in attribution for cross-lingual question answering . open-research questions are available in 20 languages, but their raw generation often falls short in factuality . |
| Approach: | They are the first to study attribution for cross-lingual question answering . they collect data in 5 languages to assess the attribution level of a state-of-the-art QA system . |
| Outcome: | The proposed approach improves the attribution level of a state-of-the-art cross-lingual QA system. |
Matching the Blanks: Distributional Similarity for Relation Learning (P19-1)
Copied to clipboard
| Challenge: | Efforts to build general purpose relation extractors that can model arbitrary relations are limited in their ability to generalize. |
| Approach: | They propose to build task-agnostic relation representations solely from entity-linked text to extend Harris’ distributional hypothesis to relations. |
| Outcome: | The proposed representations outperform previous methods on SemEval 2010 Task 8, KBP37, and TACRED even without using any of the task’s training data. |
Decontextualization: Making Sentences Stand-Alone (2021.tacl-1)
Copied to clipboard
| Challenge: | Taking excerpts of text can be problematic, as key pieces may not be explicit in a local window. |
| Approach: | They define a problem of sentence decontextualization by rewriting a sentence to be interpretable out of context while preserving its meaning. |
| Outcome: | The proposed method can be used in question answering and document understanding tasks. |
TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages (2020.tacl-1)
Copied to clipboard
Jonathan H. Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, Jennimaria Palomaki
| Challenge: | Existing models for multilingual modeling are based on a set of typological features that are used to express meaning in languages such as English. |
| Approach: | They present a question-answer-typed question-referenced dataset that covers 11 typologically diverse languages with 204K question-and-answered pairs. |
| Outcome: | The proposed dataset covers 11 typologically diverse languages with 204K question-answer pairs. |
NAIL: Lexical Retrieval Indices with Efficient Non-Autoregressive Decoders (2023.emnlp-main)
Copied to clipboard
| Challenge: | Neural document rerankers require dedicated hardware for serving, which is costly and often not feasible. |
| Approach: | They propose a method that captures 86% of the gains of a Transformer cross-attention model with a lexicalized scoring function that only requires 10-6% of . the model architecture is compatible with recent encoder-decoder and decoder-only large language models, such as T5, GPT-3 and PaLM. |
| Outcome: | The proposed model captures 86% of the gains of a Transformer cross-attention model with a lexicalized scoring function. |
1-PAGER: One Pass Answer Generation and Evidence Retrieval (2023.findings-emnlp)
Copied to clipboard
| Challenge: | 1-Pager is the first system that answers a question and retrieves evidence using a single Transformer-based model and decoding process. |
| Approach: | They propose a system that partitions the corpus using constrained decoding to select a document and answer string, and a method that uses a single Transformer-based model to generate evidence. |
| Outcome: | The proposed system outperforms the equivalent ‘closed-book’ question answering model by grounding predictions in evidence corpus. |
From RAG to Riches: Retrieval Interlaced with Sequence Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | RICHES interleaves retrieval with sequence generation tasks . traditional approaches chain LLM generation with separate retrieval model . |
| Approach: | They propose a novel approach that interleaves retrieval with sequence generation tasks . they propose attributed evidence, multi-hop retrievals and interleave thoughts to plan on what to retrieve next . |
| Outcome: | The proposed approach can work with any Instruction-tuned model, without additional training. |
Entities as Experts: Sparse Memory Access with Entity Supervision (2020.emnlp-main)
Copied to clipboard
| Challenge: | Unlike previous attempts to integrate entity knowledge into sequence models, EaE’s entity representations are learned directly from text. |
| Approach: | They propose a model that can access distinct memories of entities mentioned in a piece of text and a new architecture that can do this. |
| Outcome: | The proposed model outperforms an encoder-generator Transformer model with 10x the parameters on a task that requires 10x more parameters to answer. |