Papers by Shiwei Wu
Leveraging Entity Information for Cross-Modality Correlation Learning: The Entity-Guided Multimodal Summarization (2024.findings-acl)
Copied to clipboard
| Challenge: | Multimodal Summarization with Multimodal Output (MSMO) is a new approach to produce a multimodal summary that integrates both text and relevant images. |
| Approach: | They propose an Entity-Guided Multimodal Summarization model that integrates both text and relevant images to produce a multimodal summary. |
| Outcome: | The proposed model integrates text-image and entity-image information and refines image selection through knowledge distillation from a pre-trained vision-language model. |
Graph-to-Tree Neural Networks for Learning Structured Input-Output Translation with Applications to Semantic Parsing and Math Word Problem (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Graph2Tree model encodes graph-structured input and decodes tree-structures output. |
| Approach: | They propose a novel Graph-to-Tree Neural Network consisting of a graph encoder and a hierarchical tree decoder that encodes an augmented graph-structured input and decodes a tree-structure-output. |
| Outcome: | The proposed model outperforms or matches the performance of other state-of-the-art models on two problems, neural semantic parsing and math word problem. |