Papers by Jamie Kiros
An Empirical Study of Generation Order for Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | a recent study of generation order for machine translation shows it does not affect output quality . Neural sequence models have been successfully applied to a broad range of tasks in recent years . |
| Approach: | They propose a soft order-reward framework that enables models to follow arbitrary oracle generation policies. |
| Outcome: | The proposed framework explores a wide variety of generation orders including uninformed orders, location-based orders, frequency-based or model-based orderings, and model-driven orders. |
Illustrative Language Understanding: Large-Scale Visual Grounding with Image Search (P18-1)
Copied to clipboard
| Challenge: | a large-scale lookup operation to ground language via ‘snapshots’ of our physical world accessed through image search is currently used to learn word representations. |
| Approach: | They propose a large-scale lookup operation to ground language via ‘snapshots’ of our physical world accessed through image search. |
| Outcome: | The proposed model is based on a large-scale lookup operation to ground language using image search. |
InferLite: Simple Universal Sentence Representations from Natural Language Inference Data (D18-1)
Copied to clipboard
| Challenge: | InferLite is a lightweight version of InferSent that does not use recurrent layers and can generalize to multiple pre-trained word embeddings. |
| Approach: | They propose a lightweight version of InferSent that does not use recurrent layers and operates on a collection of pre-trained word embeddings. |
| Outcome: | The proposed model outperforms existing models that learn generic embeddings in an unsupervised setting, often requiring several days or weeks to train. |
Multichannel Generative Language Model: Learning All Possible Factorizations Within and Across Channels (2020.findings-emnlp)
Copied to clipboard
| Challenge: | MGLM is a generative joint distribution model over channels. |
| Approach: | They propose a multichannel generative joint distribution model over channels that marginalizes over all possible factorizations within and across all channels. |
| Outcome: | The proposed model outperforms traditional bilingual discriminative models. |
Generate, Annotate, and Learn: NLP with Synthetic Text (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing methods to generate unlabeled text are difficult to find. |
| Approach: | They propose a general framework called "generate, annotate, and learn" to take advantage of synthetic text within knowledge distillation, self-training, and few-shot learning applications. |
| Outcome: | The proposed framework achieves state-of-the-art knowledge distillation results for 6-layer transformers on the GLUE leaderboard. |