Papers by Matthew Henderson
TESS: Text-to-Text Self-Conditioned Simplex Diffusion (2024.eacl-long)
Copied to clipboard
Rabeeh Karimi Mahabadi, Hamish Ivison, Jaesung Tae, James Henderson, Iz Beltagy, Matthew Peters, Arman Cohan
| Challenge: | Existing models for diffusion generation are expensive and discrete, resulting in a large number of diffusion steps to generate text. |
| Approach: | They propose a text diffusion model that is fully non-autoregressive and employs a new form of self-conditioning and applies the diffusion process on the logit simplex space rather than the learned embedding space. |
| Outcome: | The proposed model outperforms state-of-the-art non-autoregressive models, requires fewer diffusion steps with minimal drop in performance, and is competitive with pretrained autoregressive sequence-to-sequence models. |
Span-ConveRT: Few-shot Span Extraction for Dialog with Pretrained Conversational Representations (2020.acl-main)
Copied to clipboard
| Challenge: | Span-ConveRT is a light-weight model for dialog slot-filling . we show consistent gains over a span extractor and a BERT-based model . |
| Approach: | They propose a model for dialog slot-filling which frames the task as a turn-based span extraction task. |
| Outcome: | The proposed model is especially useful for few-shot learning scenarios. |
Training Neural Response Selection for Task-Oriented Dialogue Systems (P19-1)
Copied to clipboard
Matthew Henderson, Ivan Vulić, Daniela Gerz, Iñigo Casanueva, Paweł Budzianowski, Sam Coope, Georgios Spithourakis, Tsung-Hsien Wen, Nikola Mrkšić, Pei-Hao Su
| Challenge: | Despite their popularity, retrieval-based models have had modest impact on task-oriented dialogue systems . main obstacle to their application is the low-data regime of most task-orientated dialogue tasks . e-commerce, banking, and other domains are applications of retrieval models . |
| Approach: | They propose a method which pretrains a retrieval-based model on large general-domain conversational corpora and fine-tunes it for the target dialogue domain. |
| Outcome: | The proposed method is evaluated on five diverse domains, ranging from e-commerce to banking. |
ConVEx: Data-Efficient and Few-Shot Slot Labeling (2021.naacl-main)
Copied to clipboard
| Challenge: | ConVEx is an efficient pretraining and fine-tuning neural approach for slot-labeling dialog tasks. |
| Approach: | They propose an efficient pretraining and fine-tuning neural approach for slot-labeling dialog tasks that uses a pairwise cloze task and reddit data. |
| Outcome: | The proposed approach is well aligned with its intended use on slot-labeling tasks and can be used across a range of domains and data sets. |
ConveRT: Efficient and Accurate Conversational Representations from Transformers (2020.findings-emnlp)
Copied to clipboard
| Challenge: | ConveRT is a pretraining framework for conversational AI that is computationally heavy, slow, and expensive to train. |
| Approach: | They propose a pretraining framework for conversational tasks that is efficient, lightweight, and inexpensive. |
| Outcome: | The proposed model achieves state-of-the-art performance across widely established responses . it trains substantially faster than existing state- of-the art models . |
PolyResponse: A Rank-based Approach to Task-Oriented Dialogue with Application in Restaurant Search and Booking (D19-3)
Copied to clipboard
Matthew Henderson, Ivan Vulić, Iñigo Casanueva, Paweł Budzianowski, Daniela Gerz, Sam Coope, Georgios Spithourakis, Tsung-Hsien Wen, Nikola Mrkšić, Pei-Hao Su
| Challenge: | a task-oriented dialogue system is based on task-specific ontologies that constrain slots to specific values . we present a conversational search engine that can be used to search for restaurant reservations . |
| Approach: | They propose a conversational search engine that supports task-oriented dialogue . the polyresponse engine is trained on hundreds of millions of examples extracted from real conversations . |
| Outcome: | The proposed system is available in 8 different languages. |