Papers by Chuanhao Lv
Incorporating Probing Signals into Multimodal Machine Translation via Visual Question-Answering Pairs (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies show that multimodal machine translation systems exhibit decreased sensitivity to visual information when text inputs are complete. |
| Approach: | They propose to generate parallel VQA style pairs from source text to foster more robust cross-modal interaction. |
| Outcome: | The proposed approach generates parallel VQA style pairs from the source text, fostering more robust cross-modal interaction. |
On Vision Features in Multimodal Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Recent work on multimodal machine translation (MMT) has focused on the way of incorporating vision features into translation but little attention is given to the quality of vision models. |
| Approach: | They develop a selective attention model to study the patch-level contribution of an image in multimodal machine translation. |
| Outcome: | The proposed model is able to learn translation from the visual modality on probing tasks and is compared with existing models. |