Papers by Haoxin Li
NarrowBERT: Accelerating Masked Language Model Pretraining and Inference (2023.acl-short)
Copied to clipboard
| Challenge: | Large-scale language model pretraining is expensive as the models and pretraining corpora have become larger over time. |
| Approach: | They propose a modified transformer encoder that increases throughput for masked language model pretraining by more than 2x. |
| Outcome: | The proposed model increases throughput on IMDB and Amazon reviews classification and CoNLL NER tasks by 3.5x with minimal performance degradation. |
Diversify, Rationalize, and Combine: Ensembling Multiple QA Strategies for Zero-shot Knowledge-based VQA (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Knowledge-based Visual Qustion-answering (K-VQA) often requires background knowledge beyond the image content. |
| Approach: | They propose a method that uses a bundle of complementary question-answering tactics to aggregate their answers using textual rationales. |
| Outcome: | Experiments show that DietCoke outperforms state-of-the-art LLM-based baselines by 2.8% and 4.7% on K-VQA. |
How Large Language Models Balance Internal Knowledge with User and Document Assertions (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models often need to balance their internal parametric knowledge with external information, such as user beliefs and content from retrieved documents, in real-world scenarios like RAG or chat-based systems. |
| Approach: | They propose a three-source interaction framework to evaluate 27 large language models from 3 families on 2 datasets. |
| Outcome: | The proposed framework systematically evaluates 27 large language models from 3 families on 2 datasets. |