Papers by Rachit Bansal
LM-CORE: Language Models with Contextually Relevant External Knowledge (2022.findings-naacl)
Copied to clipboard
| Challenge: | Large pre-trained language models can capture factual knowledge in their parameters but storing large amounts of knowledge in the model parameters is sub-optimal given the ever-growing amounts of information and resource requirements. |
| Approach: | They propose a framework that provides explicit access to contextually relevant structured knowledge to the model and train it to use that knowledge. |
| Outcome: | The proposed framework outperforms state-of-the-art knowledge-enhanced language models on knowledge probing tasks and can handle knowledge updates. |
How Low is Too Low? A Computational Perspective on Extremely Low-Resource Languages (2021.acl-srw)
Copied to clipboard
| Challenge: | Sumerian is one of the world’s oldest written languages attested from at least the beginning of the 3rd millennium BC. |
| Approach: | They propose to use interpretLR to train attention-based deep learning models in a low-resource language, Sumerian cuneiform, which includes part-of-speech tagging, named entity recognition, and machine translation. |
| Outcome: | The proposed pipeline outperforms the large language model RoBERTa for POS Tagging and NER. |
CoSe-Co: Text Conditioned Generative CommonSense Contextualizer (2022.naacl-main)
Copied to clipboard
| Challenge: | Pre-trained language models (PTLMs) have been shown to perform well on natural language tasks. |
| Approach: | They propose a commonsense contextualizer conditioned on sentences as input to make it generically usable in tasks involving natural language text. |
| Outcome: | The proposed model improves on existing methods on CSQA, ARC, QASC and OBQA datasets. |
Evaluating Explanations: How Much Do Explanations from the Teacher Aid Students? (2022.tacl-1)
Copied to clipboard
Danish Pruthi, Rachit Bansal, Bhuwan Dhingra, Livio Baldini Soares, Michael Collins, Zachary C. Lipton, Graham Neubig, William W. Cohen
| Challenge: | Existing methods to explain predictions by highlighting salient features are often unstated. |
| Approach: | They propose a framework to quantify the value of explanations via the accuracy gains that they confer on a student model trained to simulate a teacher model. |
| Outcome: | The proposed framework allows principled, automatic, model-agnostic evaluation of attributions. |