Papers by Yoshitomo Matsubara
Ensemble Transformer for Efficient and Accurate Ranking Tasks: an Application to Question Answering Systems (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Large transformer models are expensive and slow to use in many applications. |
| Approach: | They propose an efficient neural network to distill large transformers into a single smaller model. |
| Outcome: | The proposed model outperforms existing models on English datasets . it outperformed existing models with 2.7 more parameters and 2.5 slower . |
Cross-Lingual Knowledge Distillation for Answer Sentence Selection in Low-Resource Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | Cross-Lingual Knowledge Distillation (CLKD) is a method to train AS2 models for low-resource languages without labeled data. |
| Approach: | They propose a method to train AS2 models for low-resource languages without labeled data . they use a translation-based WikiQA dataset and a multilingual AS2 dataset . |
| Outcome: | The proposed method outperforms or rivals fine-tuning with labeled data and machine translation and the teacher model. |