Papers by Gerard Sant
SignCLIP: Connecting Text and Sign Language by Contrastive Learning (2024.emnlp-main)
Copied to clipboard
| Challenge: | SignCLIP is an efficient method of learning useful visual representations for sign language processing from large-scale, multilingual video-text pairs without optimizing for a specific task or sign language of limited size. |
| Approach: | They propose a method for learning visual representations for sign language processing from large-scale video-text pairs without directly optimizing for a specific task or sign language. |
| Outcome: | The proposed model can learn from multilingual video-text pairs without optimizing for a specific task or sign language of limited size. |
Multiformer: A Head-Configurable Transformer-Based Model for Direct Speech Translation (2022.naacl-srw)
Copied to clipboard
| Challenge: | Existing approaches to address speech tasks with a self-attention mechanism are expensive and lead to information loss. |
| Approach: | They propose a Transformer-based model which uses different attention mechanisms on each head to bias the self-attention towards the extraction of more diverse token interactions. |
| Outcome: | The proposed model outperforms baseline models by 0.7 BLEU in the speech task. |