Papers by Sourav Dutta
Self-Distilled Quantization: Achieving High Compression Rates in Transformer-Based Language Models (2023.acl-short)
Copied to clipboard
| Challenge: | Existing methods for quantization-aware training and quantization for learning have limitations in dealing with accumulative quantization errors. |
| Approach: | They propose a method that minimizes accumulative quantization errors and outperforms baselines by distilling knowledge from a fine-tuned teacher network. |
| Outcome: | The proposed method minimizes accumulative quantization errors and outperforms baselines on the XGLUE benchmark. |
Aligned Weight Regularizers for Pruning Pretrained Neural Networks (2022.findings-acl)
Copied to clipboard
| Challenge: | Pruning aims to reduce the number of parameters while maintaining performance close to the original network. |
| Approach: | They propose a self-distilled pruning strategy that maximizes representational similarity between pruned and unpruned networks. |
| Outcome: | The proposed pruning strategy outperforms smaller models and outperformed smaller ones with an equal number of parameters and is competitive against (6 times) larger distilled networks. |
AX-MABSA: A Framework for Extremely Weakly Supervised Multi-label Aspect Based Sentiment Analysis (2022.emnlp-main)
Copied to clipboard
| Challenge: | Aspect Based Sentiment Analysis is a dominant research area with potential applications in social media analytics, business, finance, and health. |
| Approach: | They propose a weakly supervised multi-label Aspect Category Sentiment Analysis framework which does not use any labelled data. |
| Outcome: | The proposed framework outperforms weakly supervised baselines on four benchmark datasets and is able to generate multiple aspect category-sentiment pairs per review sentence. |
Multi-Stage Framework with Refinement Based Point Set Registration for Unsupervised Bi-Lingual Word Alignment (2022.coling-1)
Copied to clipboard
| Challenge: | Existing unsupervised approaches to cross-lingual word embeddings suffer from instability and convergence issues. |
| Approach: | They propose a multi-stage framework for unsupervised mapping of bi-lingual word embeddings onto a shared vector space by combining adversarial initialization, refinement procedure and point set registration. |
| Outcome: | The proposed framework shows robustness against variable adversarial performance on diverse languages. |
Cross-lingual Sentence Embedding using Multi-Task Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing multilingual sentence embedding models require large parallel corpora to learn efficiently, limiting their scope. |
| Approach: | They propose a sentence embedding framework based on an unsupervised loss function . they capture semantic similarity and relatedness between sentences using a multi-task loss function. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on STS, BUCC and Tatoeba benchmarks and on a monolingual benchmark. |
Improved Out-of-Scope Intent Classification with Dual Encoding and Threshold-based Re-Classification (2024.lrec-main)
Copied to clipboard
| Challenge: | Current methods for intent classification often rely on assumptions about data distributions and outliers are unpredictable . |
| Approach: | They propose a dual encoder for threshold-based re-classification that generates user utterance embeddings and incorporates out-of-scope phrases from open-domain datasets. |
| Outcome: | The proposed framework outperforms benchmarks on the CLINC-150, Stackoverflow, and Banking77 datasets and achieves an increase of up to 13% and 5% in F1 score for known and unknown intents. |
AdaSent: Efficient Domain-Adapted Sentence Embeddings for Few-Shot Classification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has found that few-shot sentence classification based on pre-trained Sentence Encoders (SEs) is efficient, robust, and effective. |
| Approach: | They propose a domain-specialization approach that decouples SEPT from DAPT by training a SEPT adapter on a pre-trained PLM. |
| Outcome: | The proposed approach matches or surpasses the performance of full SEPT on DAPT-ed PLMs while significantly reducing training costs. |