Papers by Ayan Sengupta
HIT - A Hierarchically Fused Deep Attention Network for Robust Code-mixed Language Representation (2021.findings-acl)
Copied to clipboard
| Challenge: | linguistics and morphology of resource-short code-mixed texts remain a key challenge in text processing. |
| Approach: | They propose a hierarchical transformer-based framework that captures the semantic relationship among words and hierarchically learns sentencelevel semantics using a fused attention mechanism. |
| Outcome: | The proposed framework improves on one European and five Indic languages on four NLP tasks on eleven datasets. |
On the Generalization vs Fidelity Paradox in Knowledge Distillation (2025.findings-acl)
Copied to clipboard
| Challenge: | Knowledge distillation (KD) is a key technique for compressing large language models into smaller ones while preserving performance. |
| Approach: | They propose to use knowledge distillation to compress large language models into smaller ones while preserving performance. |
| Outcome: | The proposed technique improves the performance of smaller models by 10% while providing only marginal benefits for larger models. |
Manifold-Preserving Transformers are Effective for Short-Long Range Encoding (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Multi-head self-attention-based Transformers have shown promise in different learning tasks . but encoders of Transformers and their variants fail to preserve layer-wise contextual information . |
| Approach: | They propose an encoder model that guarantees a theoretical bound for layer-wise distance preservation between a pair of tokens. |
| Outcome: | The proposed model preserves equivalence between tokens and performs better than Transformers. |
Gated Transformer for Robust De-noised Sequence-to-Sequence Modelling (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Noisy texts are common in user-generated texts that appear abundant in social media platforms like SMS, online chat, email, blogs, wikis etc. |
| Approach: | They propose a sequence-to-sequence architecture that uses a gating mechanism to detect types of corrections required from English texts. |
| Outcome: | The proposed architecture performs better than non-gated models on machine translation and Summarization tasks. |