Papers by Aruna Balasubramanian
DeFormer: Decomposing Pre-trained Transformers for Faster Question Answering (2020.acl-main)
Copied to clipboard
| Challenge: | DeFormer is a transformer-based QA model that uses input-wide self-attention at all layers, causing them to be slow and memory-intensive. |
| Approach: | They propose a transformer which substitutes the full self-attention with question-wide and passage-wide self- attentions in the lower layers. |
| Outcome: | The proposed model can be used to speed up QA by over 4.3x and with simple distillation-based losses they incur only a 1% drop in accuracy. |
IrEne-viz: Visualizing Energy Consumption of Transformer Models (2021.emnlp-demo)
Copied to clipboard
Yash Kumar Lal, Reetu Singh, Harsh Trivedi, Qingqing Cao, Aruna Balasubramanian, Niranjan Balasubramanian
| Challenge: | IrEne is an energy prediction system that accurately predicts inference energy consumption of transformer-based NLP models. |
| Approach: | They present an online platform for visualizing and exploring energy consumption of transformer-based NLP models. |
| Outcome: | The proposed system predicts energy consumption of transformer-based models and their components. |
IrEne: Interpretable Energy Prediction for Transformers (2021.acl-long)
Copied to clipboard
| Challenge: | Existing software-based energy measurements of NLP models are not accurate because they do not consider the complex interactions between energy consumption and model execution. |
| Approach: | They propose an interpretable and extensible energy prediction system that predicts inference energy consumption of Transformer-based NLP models. |
| Outcome: | The proposed system predicts inference energy consumption of transformer models with an error of under 7% compared to the ground truth. |