Papers with VGG19
English to Hindi Multi-modal Neural Machine Translation and Hindi Image Captioning (D19-52)
Copied to clipboard
| Challenge: | Multi-modal translation is an emerging task of the MT community, where visual features of image combine with textual features of parallel source-target text to translate sentences. |
| Approach: | They propose to use convolutional neural net-works and visual geometry to extract image features and attention-based Neural MachineTranslation (NMT) system for translation. |
| Outcome: | The proposed multi-modal translation system improves translation quality and improves the quality of the captions of the images. |
The Effects of Unimodal Representation Choices on Multimodal Learning (L18-1)
Copied to clipboard
| Challenge: | In the real world, multiple modes of information are gathered to create knowledge in a way humans can understand. |
| Approach: | They propose to combine unimodal representations to map multiple modes of information to a single space . they argue that the way they are combined can affect performance and classification metrics . |
| Outcome: | The proposed model can be used to correlate words in a textual description of an object with multimodal representations. |