Papers by Satwik Bhattamishra
Submodular Optimization-based Diverse Paraphrasing and its Effectiveness in Data Augmentation (N19-1)
Copied to clipboard
| Challenge: | Previous work focused on generating semantically similar paraphrases without considering diversity. |
| Approach: | They propose a method to obtain highly diverse paraphrases without compromising on paraphrasing quality by using monotone submodular function maximization. |
| Outcome: | The proposed method is effective on multiple tasks such as intent classification and paraphrase recognition. |
MAGNIFICo: Evaluating the In-Context Learning Ability of Large Language Models to Generalize to Novel Interpretations (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have a knowledge cutoff and are costly to finetune repeatedly. |
| Approach: | They introduce a language evaluation suite that incorporates diverse tokens and prompt settings to simulate real-world complexity. |
| Outcome: | The proposed evaluation suite incorporates diverse tokens and prompt settings to simulate real-world complexity. |
Are NLP Models really able to Solve Simple Math Word Problems? (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing solvers for math word problems often achieve high performance on benchmark datasets . existing models rely on shallow heuristics to achieve high accuracy . |
| Approach: | They restrict their attention to English MWPs taught in grades four and lower . they propose a challenge dataset to test the accuracy of MWp solvers . |
| Outcome: | The proposed model can solve a large fraction of MWPs even with shallow heuristics . the proposed model is much lower on the challenge dataset SVAMP . |
On the Ability and Limitations of Transformers to Recognize Formal Languages (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on LSTMs have not revealed their ability to model syntactic properties. |
| Approach: | They propose to build a Transformers model for a subclass of counter languages and find that their learning mechanism strongly correlates with their construction. |
| Outcome: | The proposed model generalizes well on counter languages and its learned mechanism correlates with it. |
On the Practical Ability of Recurrent Neural Networks to Recognize Hierarchical Languages (2020.coling-main)
Copied to clipboard
| Challenge: | recurrent models have been effective in NLP tasks but performance on context-free languages (CFLs) is weak. |
| Approach: | They evaluate the performance of recurrent models on Dyck-n languages . they find that they are expressive enough to recognize Dyck words of arbitrary lengths if their depths are bounded. |
| Outcome: | The proposed models generalize well on Dyck-n languages, while performing poorly on longer test strings. |
Revisiting the Compositional Generalization Abilities of Neural Sequence Models (2022.acl-short)
Copied to clipboard
| Challenge: | Existing studies have suggested that standard seq-to-seq models lack the ability to generalize compositionally. |
| Approach: | They propose to use one-shot primitive generalization as introduced by the popular SCAN benchmark to modify the training distribution in simple and intuitive ways to achieve near-perfect generalization performance. |
| Outcome: | The proposed model achieves near-perfect generalization performance despite a lack of training data . |
Simplicity Bias in Transformers and their Ability to Learn Sparse Boolean Functions (2023.acl-long)
Copied to clipboard
| Challenge: | Recent studies have found that Transformers struggle to model several formal languages when compared to recurrent models. |
| Approach: | They conduct an extensive empirical study on Boolean functions to demonstrate that Transformers are relatively more biased towards functions of low sensitivity . they also show that Transformer's generalize near perfectly even in the presence of noisy labels whereas recurrent models overfit and achieve poor generalization accuracy. |
| Outcome: | The results show that Transformers generalize near perfectly even in noisy Boolean functions whereas recurrent models overfit and achieve poor generalization accuracy. |