Papers with SNI
MoDE: Effective Multi-task Parameter Efficient Fine-Tuning with a Mixture of Dyadic Experts (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent efforts have explored mixtures of LoRA modules for multi-task settings, but this study reveals redundancy in the down-projection matrix of these architectures. |
| Approach: | They propose a method to share down-projection matrix across tasks and employ atomic rank-one adapters coupled with routers that allow more sophisticated task-level specialization. |
| Outcome: | The proposed method outperforms state-of-the-art models on a SNI benchmark and provides a practical solution for deploying lightweight models. |
Interventional Speech Noise Injection for ASR Generalizable Spoken Language Understanding (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to increase the robustness of pre-trained language models (PLMs) against unseen ASR systems produce noisy inputs for SLU models, which can significantly degrade their performance. |
| Approach: | They propose to introduce ASR-plausible noises into pre-trained language models by cutting off the non-causal effect of noises. |
| Outcome: | The proposed method improves the robustness and generalizability of SLU models against unseen ASR systems by cutting off the non-causal effect of noises. |