Papers by Joelle Pineau
Extending Neural Generative Conversational Model using External Knowledge Sources (D18-1)
Copied to clipboard
| Challenge: | Existing generative dialogue models lack coherence and are content poor . however, current models lack the capacity to handle large unstructured knowledge sources. |
| Approach: | They propose an architecture to incorporate unstructured knowledge sources to enhance the next utterance prediction in chit-chat type of generative dialogue models. |
| Outcome: | The proposed architecture improves the next utterance prediction in chit-chat type of generative dialogue models by incorporating external knowledge from Wikipedia summaries and the NELL knowledge base. |
Questions Are All You Need to Train a Dense Passage Retriever (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing methods for dense retrieval require large supervised datasets with custom hard-negative mining and denoising of positive examples. |
| Approach: | They propose a new corpus-level autoencoding approach for training dense retrieval models that does not require labeled training data. |
| Outcome: | The proposed method matches or surpasses strong supervised performance levels on multiple QA benchmarks with no labeled training data or task-specific losses. |
CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text (D19-1)
Copied to clipboard
| Challenge: | Existing datasets for reading comprehension tasks have been used to test the generalization of natural language understanding systems. |
| Approach: | They propose a diagnostic benchmark suite to clarify key issues related to the robustness and systematicity of NLU systems. |
| Outcome: | The proposed benchmark suite clarifies key issues related to the robustness and systematicity of NLU systems. |
Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little (2021.emnlp-main)
Copied to clipboard
| Challenge: | masked language models (MLMs) pre-train to model higher-order word co-occurrence statistics . authors suggest that such models have learned to represent syntactic structures prevalent in classical NLP pipelines . purely distributional information largely explains the success of pre-training, authors say . |
| Approach: | They propose to pre-train masked language models on sentences with random shuffled word order and show they still achieve high accuracy after fine-tuning on many downstream tasks. |
| Outcome: | The proposed model performs well according to parametric syntactic probes . the authors argue that the model is not all that different from earlier distributional models . |
Improving Passage Retrieval with Zero-Shot Question Generation (2022.emnlp-main)
Copied to clipboard
Devendra Sachan, Mike Lewis, Mandar Joshi, Armen Aghajanyan, Wen-tau Yih, Joelle Pineau, Luke Zettlemoyer
| Challenge: | Existing re-ranking methods for open-domain question answering are not domain- or task-specific. |
| Approach: | They propose a simple and effective re-ranking method for improving passage retrieval in open-domain question answering. |
| Outcome: | The proposed method outperforms strong supervised models on open-domain questions and triviaQA datasets on top-1000 passages. |
Exploring the Limits of Few-Shot Link Prediction in Knowledge Graphs (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for few-shot link prediction are limited by having only a few examples of a relation . low-frequency relations are abundant in knowledge graphs, but link prediction for these relations is important . |
| Approach: | They perform few-shot link prediction for a set of new relations unseen during training, given only a few examples of each relation at test time. |
| Outcome: | The proposed model is based on a simple, zero-shot baseline that ignores relation-specific information and achieves surprisingly strong performance. |
Learning an Unreferenced Metric for Online Dialogue Evaluation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing tools for dialogue evaluation do not generalize to unseen datasets and/or need a human-generated reference response during inference. |
| Approach: | They propose an unreferenced automated dialogue evaluation metric that uses large pre-trained language models to extract latent representations of utterances and leverages the temporal transitions that exist between them. |
| Outcome: | The proposed model achieves higher correlation with human annotations in an online setting, while not requiring true responses for comparison during inference. |
The Curious Case of Absolute Position Embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | In natural language, it is not absolute position that matters, but relative position . et al., 2017) language models incorporate positional encodings that encode absolute (linear) word order. |
| Approach: | They find that Transformer language models encode word order using positional information . they also find that models that use absolute position embeddings over-rely on positional data . |
| Outcome: | The results raise questions about the efficacy of APEs to model the relativity of position information. |
Seeded self-play for language learning (D19-64)
Copied to clipboard
| Challenge: | Current methods for learning human language are too data inefficient to learn it in this way. |
| Approach: | They propose to train a meta-learning agent in simulation to interact with populations of pre-trained agents, each with their own distinct communication protocol. |
| Outcome: | The proposed algorithm minimizes the number of on-policy interactions while learning human language while minimizing the number on-political interactions. |
Sometimes We Want Ungrammatical Translations (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in Neural Machine Translation (NMT) systems focus on improving translation quality and improving robustness to perturbations. |
| Approach: | They propose a way to quantify faithfulness to the original text by focusing on word-order perturbations. |
| Outcome: | The proposed method aims to measure faithfulness and robustness in word-order perturbations without deleting or injecting tokens. |
UnNatural Language Inference (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that pre-trained NLU models understand human-like syntax . however, these models are word order invariant, causing them to assign gold labels to permutations . |
| Approach: | They propose to measure the severity of this issue by examining the properties of particular permutations that lead models to be word order invariant. |
| Outcome: | The proposed model is word order invariant, but it's not human-like syntax. |