Papers by Shrimai Prabhumoye
Topological Sort for Sentence Ordering (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work on sentence ordering task has framed it as a sequence prediction problem. |
| Approach: | They propose a new constraint solving problem and propose 'human evaluation' they propose to capture coherence in documents by arranging sentences in the correct order . |
| Outcome: | The proposed technique captures coherence in documents better than previous approaches. |
Context Generation Improves Open Domain Question Answering (2023.findings-eacl)
Copied to clipboard
Dan Su, Mostofa Patwary, Shrimai Prabhumoye, Peng Xu, Ryan Prenger, Mohammad Shoeybi, Pascale Fung, Anima Anandkumar, Bryan Catanzaro
| Challenge: | Existing closed-book question answering methods do not fully exploit the parameterized knowledge. |
| Approach: | They propose a closed-book QA framework which uses a coarse-to-fine approach to extract the relevant knowledge and answer a question. |
| Outcome: | The proposed method outperforms open-book QA methods on three QA benchmarks. |
LLM-Evolve: Evaluation for LLM’s Evolving Capability on Benchmarks (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks for large language models evaluate LLMs on i.i.d. tasks, overlooking their ability to learn iteratively from past experiences. |
| Approach: | They propose a framework which extends established benchmarks to sequential problem-solving settings and provides feedback after each round to build a demonstration memory that the models can query in future tasks. |
| Outcome: | The proposed framework can improve performance of LLMs by learning from past interactions and improve models' performance over time. |
Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoning (2026.eacl-long)
Copied to clipboard
Syeda Nahida Akter, Shrimai Prabhumoye, Matvei Novikov, Seungju Han, Ying Lin, Evelina Bakhturina, Eric Nyberg, Yejin Choi, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
| Challenge: | Prior work has successfully applied Reinforcement Learning (RL) to mathematical reasoning, but generalization to broader domains remains challenging due to limited data and lack of verifiable rewards for unstructured domains. |
| Approach: | They propose a framework that integrates multi-domain corpora into RL training to improve generalization across diverse reasoning tasks. |
| Outcome: | The proposed framework improves generalization across diverse reasoning tasks. |
Focused Attention Improves Document-Grounded Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Document grounded generation is the task of using the information provided in a document to improve text generation. |
| Approach: | They propose two new document grounded generation tasks that use information provided in a document to improve text generation. |
| Outcome: | The proposed models outperform existing methods on automated and human evaluation for closeness to reference and relevance to the document. |
Towards Content Transfer through Grounded Text Generation (N19-1)
Copied to clipboard
| Challenge: | Recent work in neural natural language generation has attracted significant interest in controlling the form of text, such as style, persona, and wordiness. |
| Approach: | They propose a task where the task is to generate a next sentence in a document that fits its context and is grounded in . external textual source such as a news story. |
| Outcome: | The proposed task is based on 640k Wikipedia referenced sentences paired with the source articles to show significant improvements against baselines. |
Data, Data Everywhere: A Guide for Pretraining Dataset Construction (2024.emnlp-main)
Copied to clipboard
Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings, Bo Liu, Aastha Jhunjhunwala, Zhilin Wang, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
| Challenge: | Recent language models have impressive capabilities on a number of evaluation areas. |
| Approach: | They conduct systematic analysis of pretraining set construction to identify which methods yield the greatest gains in model accuracy. |
| Outcome: | The proposed method can be used to refine and improve a pretraining set. |
Style Transfer Through Back-Translation (P18-1)
Copied to clipboard
| Challenge: | a new method for automatic style transfer is proposed to preserve the meaning of the text while reducing stylistic properties. |
| Approach: | They propose a method for automatic style transfer that uses latent representations of the input sentence to preserve meaning while reducing stylistic properties. |
| Outcome: | The proposed method improves on sentiment, gender and political slant styles on three different styles. |
Adding Instructions during Pretraining: Effective way of Controlling Toxicity in Language Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Pretrained large language models generate harmful language encompassing hate speech, abusive language, social biases, and threats. |
| Approach: | They propose two strategies that augment pretraining data to reduce model toxicity . MEDA adds raw toxicity score as meta-data and INST adds instructions indicating toxicity to pretraining samples. |
| Outcome: | The proposed strategies reduce toxicity probability up to 61% while preserving accuracy on five benchmark NLP tasks and improving AUC scores on bias detection tasks by 1.3%. |
Exploring Controllable Text Generation Techniques (2020.coling-main)
Copied to clipboard
| Challenge: | Neural controllable text generation has a plethora of applications but there is no unifying theme. |
| Approach: | They propose a new schema for the control of attributes in the generation process by classifying it into five modules and providing an analysis on the advantages and disadvantages of these techniques. |
| Outcome: | The proposed frameworks can be used to control the attributes of natural sentences and to modulate the formality and politeness of emails. |
Evaluating Parameter Efficient Learning for Generation (2022.emnlp-main)
Copied to clipboard
Peng Xu, Mostofa Patwary, Shrimai Prabhumoye, Virginia Adams, Ryan Prenger, Wei Ping, Nayeon Lee, Mohammad Shoeybi, Bryan Catanzaro
| Challenge: | Parameter efficient learning methods (PERMs) are gaining attention for their ability to adapt to a downstream task. |
| Approach: | They propose to use parameter efficient learning methods to improve model adaptation . they compare in-domain evaluations and generalizations to unseen domains and new datasets . |
| Outcome: | The proposed method outperforms finetuning and PERMs in in-domain evaluations. |
Multi-Stage Prompting for Knowledgeable Dialogue Generation (2022.findings-acl)
Copied to clipboard
Zihan Liu, Mostofa Patwary, Ryan Prenger, Shrimai Prabhumoye, Wei Ping, Mohammad Shoeybi, Bryan Catanzaro
| Challenge: | Existing knowledge-grounded dialogue systems typically use finetuned versions of a pretrained language model and large-scale knowledge bases. |
| Approach: | They propose a multi-stage prompting approach to generate knowledgeable responses from a single pretrained LM. |
| Outcome: | The proposed model outperforms the state-of-the-art retrieval-based model in terms of knowledge relevance and correctness by 5.8% and 5%, respectively. |
A Dataset for Document Grounded Conversations (D18-1)
Copied to clipboard
| Challenge: | a dataset of document grounded conversations provides information on content of a document . current datasets lacking conversation grounding do not provide this information . |
| Approach: | They propose a document grounded dataset for conversations . they use Wikipedia articles about popular movies to define document grounded conversations based on their results . |
| Outcome: | The proposed dataset provides a source of information and provides benchmark performance on the task of generating the next response. |
Politeness Transfer: A Tag and Generate Approach (2020.acl-main)
Copied to clipboard
Aman Madaan, Amrith Setlur, Tanmay Parekh, Barnabas Poczos, Graham Neubig, Yiming Yang, Ruslan Salakhutdinov, Alan W Black, Shrimai Prabhumoye
| Challenge: | Prior work on text style transfer has not focused on politeness as a style transfer task and we argue that defining it is cumbersome. |
| Approach: | They propose a task of politeness transfer which involves converting non-polite sentences to polite sentences while preserving the meaning. |
| Outcome: | The proposed model outperforms state-of-the-art methods on content preservation and style transfer accuracy. |
Case Study: Deontological Ethics in NLP (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work in natural language processing (NLP) has focused on ethical challenges . ethical foundations of NLP systems have not been explored . |
| Approach: | They propose to use deontological ethics to analyze ethical issues in natural language processing from the perspective of NLP. |
| Outcome: | The proposed ethical frameworks are based on the generalization principle and respect for autonomy through informed consent. |