Papers by Weili Liu
Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning (2023.findings-emnlp)
Copied to clipboard
Zhuolin Yang, Wei Ping, Zihan Liu, Vijay Korthikanti, Weili Nie, De-An Huang, Linxi Fan, Zhiding Yu, Shiyi Lan, Bo Li, Mohammad Shoeybi, Ming-Yu Liu, Yuke Zhu, Bryan Catanzaro, Chaowei Xiao, Anima Anandkumar
| Challenge: | Existing methods for image-to-text generation store all knowledge within parameters, thus requiring computational-expensive fine-tuning. |
| Approach: | They propose a Retrieval-augmented Visual Language Model that stores all the knowledge within parameters and can be used to retrieve it from the external database. |
| Outcome: | The proposed model significantly boosts performance for image-to-text generation tasks with 4x less parameters compared with baseline methods. |
HiPrune: Hierarchical Attention for Efficient Token Pruning in Vision-Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for visual token pruning lack insight into the intrinsic property of the vision encoder . et al., 2017: 99.3% of task accuracy with only 1/3 of the tokens. |
| Approach: | They propose a model-agnostic token pruning method that trains without training . they propose 'HiPrune' method which prunes visual tokens according to their attention . |
| Outcome: | The proposed method achieves 99.3% of task accuracy with only 1/3 of the tokens . it reduces inference FLOPs by 58.7% and maintains 99.99% accuracy with 2/9 tokens. |
COVID-19 Literature Knowledge Graph Construction and Drug Repurposing Report Generation (2021.naacl-demos)
Copied to clipboard
Qingyun Wang, Manling Li, Xuan Wang, Nikolaus Parulian, Guangxing Han, Jiawei Ma, Jingxuan Tu, Ying Lin, Ranran Haoran Zhang, Weili Liu, Aabhas Chauhan, Yingjun Guan, Bangzheng Li, Ruisong Li, Xiangchen Song, Yi Fung, Heng Ji, Jiawei Han, Shih-Fu Chang, James Pustejovsky, Jasmine Rah, David Liem, Ahmed ELsayed, Martha Palmer, Clare Voss, Cynthia Schneider, Boyan Onyshkevych
| Challenge: | a new framework to digest relevant biomedical knowledge is needed to combat COVID-19 . quantity of research results is a bottleneck, and false information promoted in publications . |
| Approach: | a team of researchers has developed a framework to extract multimedia knowledge elements from scientific literature to combat COVID-19. |
| Outcome: | a new framework extracts fine-grained multimedia knowledge elements from scientific literature . it provides detailed contextual sentences, subfigures, and knowledge subgraphs as evidence . the framework is based on a case study of drug repurposing . |
EVIDENCEMINER: Textual Evidence Discovery for Life Sciences (2020.acl-demos)
Copied to clipboard
Xuan Wang, Yingjun Guan, Weili Liu, Aabhas Chauhan, Enyi Jiang, Qi Li, David Liem, Dibakar Sigdel, John Caufield, Peipei Ping, Jiawei Han
| Challenge: | EVIDENCEMINER is a web-based system that allows users to query a natural language statement and retrieve textual evidence from a background corpora for life sciences. |
| Approach: | They propose a web-based system that lets users query a natural language statement and automatically retrieves textual evidence from a background corpora for life sciences. |
| Outcome: | EVIDENCEMINER is a web-based system that lets users query a natural language statement and automatically retrieves textual evidence from a background corpora for life sciences. |