Papers by Siddhartha Brahma
CLAR: A Cross-Lingual Argument Regularizer for Semantic Role Labeling (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for training one model on multiple languages outperform monolingual baselines for low resource languages. |
| Approach: | They propose a method to combine training data from multiple languages to create a shared representation space for the model. |
| Outcome: | The proposed method outperforms monolingual and polyglot training on low resource languages. |
Learning Explainable Linguistic Expressions with Neural Inductive Logic Programming for Sentence Classification (2020.emnlp-main)
Copied to clipboard
Prithviraj Sen, Marina Danilevsky, Yunyao Li, Siddhartha Brahma, Matthias Boehm, Laura Chiticariu, Rajasekar Krishnamurthy
| Challenge: | Existing approaches to explain models are difficult to interpret and have undesirable biases. |
| Approach: | They propose a neural network architecture for learning transparent sentences . they use linguistic expressions built on top of predicates extracted using shallow natural language understanding . |
| Outcome: | The proposed model outperforms statistical relational learning and other neuro-symbolic methods and performs better than black-box recurrent neural networks. |
Small but Mighty: New Benchmarks for Split and Rephrase (2020.emnlp-main)
Copied to clipboard
| Challenge: | Split and Rephrase is a text simplification task that requires a strong evaluation benchmark and metric . despite its relatively new nature, the benchmark dataset contains easily exploitable syntactic cues . |
| Approach: | They propose to use crowdsourced datasets to evaluate split and rephrase models . they find that the widely used benchmark dataset universally contains exploitable syntactic cues . |
| Outcome: | The proposed model performs better than the state-of-the-art model, the authors say . they show that the datasets contain significantly more diverse syntax . |
CoLT5: Faster Long-Range Transformers with Conditional Computation (2023.emnlp-main)
Copied to clipboard
Joshua Ainslie, Tao Lei, Michiel de Jong, Santiago Ontanon, Siddhartha Brahma, Yury Zemlyanskiy, David Uthus, Mandy Guo, James Lee-Thorp, Yi Tay, Yun-Hsuan Sung, Sumit Sanghai
| Challenge: | Many natural language processing tasks require long inputs, but processing long documents with a Transformer model is expensive due to quadratic attention complexity and applying feedforward and attention projection layers to every input token. |
| Approach: | They propose a long-input Transformer model that builds on the intuition that some tokens are more important than others and uses conditional computation to devote more computation to important tokens. |
| Outcome: | The proposed model achieves stronger performance than LongT5 with faster training and inference, achieving SOTA on the long-input SCROLLS benchmark. |
Improved Language Modeling by Decoding the Past (P19-1)
Copied to clipboard
| Challenge: | Existing methods to improve language modeling performance are based on regularized LSTMs with a large number of parameters and training time. |
| Approach: | They propose a method that decodes the last token in context using the predicted distribution of the next token. |
| Outcome: | The proposed method improves perplexity on the Penn Treebank dataset by 1.8 points and 2.3 points on the WikiText-2 datasets. |