Papers by Quang Pham
Who’s Who: Large Language Models Meet Knowledge Conflicts in Practice (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent large-scale pretrained language models excel in tasks requiring natural language understanding, but they often "hallucinate" plausible but incorrect content due to outdated or incorrect pretraining information. |
| Approach: | They propose a public benchmark dataset to examine model’s behavior in knowledge conflict situations. |
| Outcome: | The proposed model induces conflicts by asking about a common property among entities having the same name, resulting in questions with up to 8 distinctive answers. |
HyperRouter: Towards Efficient Training and Inference of Sparse Mixture of Experts (2023.emnlp-main)
Copied to clipboard
Truong Do, Le Khiem, Quang Pham, TrungTin Nguyen, Thanh-Nam Doan, Binh Nguyen, Chenghao Liu, Savitha Ramasamy, Xiaoli Li, Steven Hoi
| Challenge: | Recent studies suggest that fixing the routers can achieve competitive performance by alleviating the collapsing problem, where all experts eventually learn similar representations. |
| Approach: | They propose a method that dynamically generates router parameters through a fixed hypernetwork and trainable embeddings to achieve a balance between training the routers and freezing them to learn an improved routing policy. |
| Outcome: | Experiments on a wide range of tasks show that the proposed method performs better than existing methods. |