Papers by Sam Wiseman
A Multi-Task Approach for Disentangling Syntax and Semantics in Sentence Representations (N19-1)
Copied to clipboard
| Challenge: | Empirically, the model with the best performing syntactic and semantic representations gives rise to the most disentangled representations. |
| Approach: | They propose a generative model that uses latent variables to learn a sentence that uses both latent and latent representations. |
| Outcome: | The proposed model achieves better disentanglement between semantic and syntactic representations by training with multiple losses, including losses that exploit aligned paraphrastic sentences and word-order information. |
ENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | a non-autoregressive machine translation model can minimize the autoregressive teacher's energy . engINE is an inference network trained to minimize the teacher' energy based on distilled corpora . |
| Approach: | They propose to train a non-autoregressive machine translation model to minimize autoregressive teacher energy by using an inference network instead of distilled corpora. |
| Outcome: | The proposed model achieves state-of-the-art non-autoregressive results on two datasets . the proposed model is trained to minimize the autoregressive teacher energy . |
Learning Neural Templates for Text Generation (D18-1)
Copied to clipboard
| Challenge: | Encoder-decoder models are uninterpretable and difficult to control in terms of content. |
| Approach: | They propose a neural generation system using a hidden semi-markov model which learns latent templates jointly with learning to generate. |
| Outcome: | The proposed model learns useful templates and achieves strong performance nearing that of encoder-decoder models. |
Data-to-text Generation by Splicing Together Nearest Neighbors (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on data-to-text generation relies on retrieved "neighbors" but instead generates text token-by-token, left-to right. |
| Approach: | They propose to splice together retrieved segments of text from "neighbor" source-target pairs to generate text token-by-token, left-to-right. |
| Outcome: | The proposed method performs on par with strong baselines in terms of automatic and human evaluation, but allows for more interpretable and controllable generation. |
Seq2seq is All You Need for Coreference Resolution (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on coreference resolution suggests task-specific models are necessary . a recent line of work that take an alternative approach leveraging advances in seq2seq-based models is needed . |
| Approach: | They propose a pretrained seq2seq transformer to map an input document to a tagged sequence encoding the coreference annotation. |
| Outcome: | The proposed model outperforms or matches the best coreference systems on an array of datasets. |
Controllable Paraphrase Generation with a Syntactic Exemplar (P19-1)
Copied to clipboard
| Challenge: | Prior work on controllable text generation assumes that the generated attribute can take on a finite set of values known a priori. |
| Approach: | They propose a task where the syntax of a generated sentence is controlled rather by a sentential exemplar. |
| Outcome: | The proposed model achieves improvements over baselines and learns to capture desirable characteristics. |
The Mechanical Bard: An Interpretable Machine Learning Approach to Shakespearean Sonnet Generation (2023.acl-short)
Copied to clipboard
| Challenge: | Rather than train a model to obey these constraints implicitly, we opt to enforce them explicitly using a simple but novel approach to generation. |
| Approach: | They propose to automate the generation of sonnets within preset poetic constraints using a constrained decoding approach that uses a relatively modest neural backbone. |
| Outcome: | The proposed method produces sonnets that adhere to the genre’s defined constraints and contain lyrical language and literary devices. |
WikiTableT: A Large-Scale Data-to-Text Dataset for Generating Wikipedia Article Sections (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets for data-to-text generation focus on single-sentence generation or long-form generation. |
| Approach: | They create a dataset that pairs Wikipedia sections with tabular data and various metadata. |
| Outcome: | The proposed dataset can generate fluent and high quality texts but struggle with coherence and factuality. |
Approximating CKY with Transformers (2023.findings-emnlp)
Copied to clipboard
| Challenge: | CKY algorithm is a cubic dependence on sentence length, but transformers can be used to approximate it. |
| Approach: | They propose a transformer-based approach that approximates the CKY algorithm by directly predicting a sentence's parse and avoiding its cubic dependence on sentence length. |
| Outcome: | The proposed approach achieves better performance than comparable parsers that make use of CKY, while being faster. |
SummScreen: A Dataset for Abstractive Screenplay Summarization (2022.acl-long)
Copied to clipboard
| Challenge: | Existing summarization datasets are constructed from various domains, such as news, and we characterize them using two entity-centric metrics. |
| Approach: | They propose to use a summarization dataset to evaluate TV series transcripts and recaps . they propose to employ two entity-centric metrics to evaluate the dataset . |
| Outcome: | The proposed model outperforms the existing model and its oracle counterparts in character overlap and accuracy. |
Baked-in State Probing (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work shows language models trained on form can capture aspects of meaning without explicit state supervision. |
| Approach: | They propose to use probing to "bake" state knowledge into language models . they propose to probe for underlying world state knowledge via text prompts . |
| Outcome: | The proposed methods show that language models trained on form can capture the world state without state supervision. |
Deep Latent Variable Models of Natural Language (D18-3)
Copied to clipboard
| Challenge: | In this tutorial, we will discuss the challenges of applying neural variational inference to NLP problems. |
| Approach: | The tutorial will cover deep latent variable models in the case where exact inference over the latent variables is tractable. |
| Outcome: | The proposed tutorial will cover deep latent variable models in the case where inference cannot be performed tractably and when it is not . |
Entity Tracking Improves Cloze-style Reading Comprehension (D18-1)
Copied to clipboard
| Challenge: | Recent work on reading comprehension tasks has improved with simple approaches, but still trail human performance. |
| Approach: | They propose to add additional entity features and a multi-task tracking objective to improve model performance . they compare the model's predictions with those of more complicated models . |
| Outcome: | The proposed model outperforms the current state of the art on the LAMBADA dataset by 8 pts. |
Discrete Latent Variable Representations for Low-Resource Text Classification (2020.acl-main)
Copied to clipboard
| Challenge: | Several approaches to learning discrete latent variable models for text are available. |
| Approach: | They compare several approaches to learning discrete latent variable models for text in the case where exact marginalization over these variables is intractable. |
| Outcome: | The learned models outperform the previous best models in low-resource settings while learning significantly more compressed representations. |
Sequence Reducible Holdout Loss for Language Model Pretraining (2024.lrec-main)
Copied to clipboard
| Challenge: | Data selection techniques have shown empirical benefits in reducing the number of gradient steps to train neural models. |
| Approach: | They propose to modify an existing data selection technique to adapt it to the sequence losses typical in language modeling. |
| Outcome: | The proposed technique reduces the number of steps required to train neural models by 4.3% and improves generalization ability on out of domain datasets. |
Learning to Ignore: Long Document Coreference with Bounded Memory Neural Networks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current models for document coreference resolution have large memory requirements and quadratic runtime in document length. |
| Approach: | They propose a memory-augmented neural network that tracks only a small number of entities at a time. |
| Outcome: | The proposed model outperforms existing models on OntoNotes and LitBank in memory management and memory management. |
Label-Agnostic Sequence Labeling by Copying Nearest Neighbors (P19-1)
Copied to clipboard
| Challenge: | Retrieve-and-edit based structured prediction models condition on retrieved neighbors to generate new structures, but many models do not explicitly capture the discrete operations that allow for the neighbors to be edited into the target structure. |
| Approach: | They propose to explicitly condition on retrieved neighbors to create new structures . they propose to use a dynamic programming approach to sequence labeling . |
| Outcome: | The proposed model can perform accurate sequence labeling by explicitly copying labels from retrieved neighbors. |