Papers by Tushar Vaidya
Adapter Pruning using Tropical Characterization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on adapter pruning have not examined the optimal number of adapter parameters needed for downstream applications. |
| Approach: | They propose an adapter pruning approach that prunes adapter parameters without changing the orientation of underlying tropical hypersurfaces. |
| Outcome: | The proposed approach prunes adapter layers without changing the orientation of underlying tropical hypersurfaces. |
KNOT: Knowledge Distillation Using Optimal Transport for Solving NLP Tasks (2022.coling-1)
Copied to clipboard
| Challenge: | Knowledge Distillation using Optimal Transport (KNOT) aims to distill the natural language semantic knowledge from multiple teacher networks to a student network. |
| Approach: | They propose to distill natural language semantic knowledge from multiple teacher networks to a student network by learning to minimize the optimal transport cost of its assigned probability distribution over the labels to the weighted sum of probabilities predicted by the (local) teacher models. |
| Outcome: | The proposed method shows improvements in the global model’s SD performance over the baseline across three NLP tasks while performing on par with Entropy-based distillation on standard accuracy and F1 metrics. |