Papers by Pramuditha Perera
Benchmarking Diverse-Modal Entity Linking with Generative Models (2023.findings-acl)
Copied to clipboard
Sijia Wang, Alexander Hanbo Li, Henghui Zhu, Sheng Zhang, Pramuditha Perera, Chung-Wei Hang, Jie Ma, William Yang Wang, Zhiguo Wang, Vittorio Castelli, Bing Xiang, Patrick Ng
| Challenge: | Existing models for diverse-mode entity linking (EL) work well on per modality configurations, but it is more challenging to design a unified model for diverse modality. |
| Approach: | They propose a generative diverse-modal model that integrates text, image and table . they propose combining a multimodal encoder-decoder paradigm with a fine-tuning GDMM . |
| Outcome: | The proposed model outperforms state-of-the-art models by 8.51 F1 on average for diverse-modal EL. |
Generate then Select: Open-ended Visual Question Answering Guided by World Knowledge (2023.findings-acl)
Copied to clipboard
Xingyu Fu, Sheng Zhang, Gukyeong Kwon, Pramuditha Perera, Henghui Zhu, Yuhao Zhang, Alexander Hanbo Li, William Yang Wang, Zhiguo Wang, Vittorio Castelli, Patrick Ng, Dan Roth, Bing Xiang
| Challenge: | Open-ended Visual Question Answering (VQA) requires models to reason over visual and natural language inputs using world knowledge. |
| Approach: | They propose a new VQA pipeline that deploys a generate-then-select strategy guided by world knowledge for the first time. |
| Outcome: | The proposed pipeline expands the knowledge coverage from in-domain training data by 4.1% on OK-VQA, without additional computation cost. |