Papers by Raul Puri
Large Scale Multi-Actor Generative Dialog Modeling (2020.acl-main)
Copied to clipboard
| Challenge: | Non-goal oriented dialog agents typically exhibit inconsistent personality across conversations or the average personality of all users. |
| Approach: | They propose a model that conditionally models past conversations to probabilistically model multi-turn conversations in the actor’s persona. |
| Outcome: | The proposed model improves perplexity on 1.7M held out Reddit conversations by 0.47 on scaling from 117M to 8.3B parameters. |
MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models (2020.emnlp-main)
Copied to clipboard
Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Raul Puri, Pascale Fung, Anima Anandkumar, Bryan Catanzaro
| Challenge: | Existing pre-trained large language models have shown unparalleled generative capabilities, but they are not controllable. |
| Approach: | They propose a framework that uses large-scale language models and adds control to text generation by incorporating an external knowledge base. |
| Outcome: | The proposed model generates more fluent, consistent, and coherent stories with less repetition and higher diversity compared to previous work on the ROC story dataset. |
Training Question Answering Models From Synthetic Data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on question and answer generation aims to improve question answering models given limited amount of labeled data. |
| Approach: | They synthesize questions and answers from a synthetic text corpus generated by an 8.3 billion parameter GPT-2 model and achieve 88.4 Exact Match (EM) and 93.9 F1 score on the SQuAD1.1 dev set. |
| Outcome: | The proposed model achieves higher accuracy than the SQUAD1.1 training set questions using synthetic questions and answers than the training set question. |
BioMegatron: Larger Biomedical Domain Language Model (2020.emnlp-main)
Copied to clipboard
Hoo-Chang Shin, Yang Zhang, Evelina Bakhturina, Raul Puri, Mostofa Patwary, Mohammad Shoeybi, Raghav Mani
| Challenge: | Existing studies on domain language models do not study the factors affecting performance on domain languages. |
| Approach: | They empirically evaluate factors that can affect performance on domain language applications . sub-word vocabulary set, model size, pre-training corpus, and domain transfer are important . |
| Outcome: | The results show language models trained on biomedical text perform better on biomedicine benchmarks than those trained on general domain text corpora. |