Papers by KaShun Shum
RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Retrieval-augmented generation (RAG) is a main technique for alleviating hallucinations in large language models. |
| Approach: | They propose to integrate RAG into large language models to analyze word-level hallucinations using a corpus of 18,000 naturally generated responses from diverse LLMs. |
| Outcome: | The proposed model can fine tune a relatively small LLM and achieve a competitive hallucination detection performance when compared to the existing prompt-based approaches. |
Unmasking Deceptive Visuals: Benchmarking Multimodal Large Language Models on Misleading Chart Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Misleading visualizations can distort perception and lead to incorrect conclusions. |
| Approach: | They propose a large-scale multimodal dataset to evaluate MLLMs on misleading chart reasoning. |
| Outcome: | The proposed framework evaluates MLLMs on misleading chart reasoning on a large-scale multimodal dataset spanning 21 misleader types and 10 chart types . it contains 3,026 curated examples spanning standard chart code, CSV data, multiple-choice questions, and labeled explanations, validated through iterative MLML checks and exhausted expert human review. |
Plum: Prompt Learning using Metaheuristics (2024.findings-acl)
Copied to clipboard
Rui Pan, Shuo Xing, Shizhe Diao, Wenhe Sun, Xiang Liu, KaShun Shum, Jipeng Zhang, Renjie Pi, Tong Zhang
| Challenge: | Recent advances in prompt learning have led to a need for general prompt optimization methods. |
| Approach: | They propose a branch of discrete non-convex optimization methods with over 100 options as a promising approach to prompt learning. |
| Outcome: | The proposed methods can be used to discover more human-understandable prompts that were previously unknown in reasoning and image generation tasks. |
LMFlow: An Extensible Toolkit for Finetuning and Inference of Large Foundation Models (2024.naacl-demo)
Copied to clipboard
| Challenge: | Foundation models have demonstrated a great ability to achieve general human-level intelligence far beyond traditional approaches. |
| Approach: | They propose a toolkit to simplify the finetuning of general foundation models. |
| Outcome: | The proposed toolkit simplifies the domain- and task-aware finetuning of general foundation models with limited computing resources. |
FIRST: Teach A Reliable Large Language Model Through Efficient Trustworthy Distillation (2024.emnlp-main)
Copied to clipboard
KaShun Shum, Minrui Xu, Jianshu Zhang, Zixin Chen, Shizhe Diao, Hanze Dong, Jipeng Zhang, Muhammad Raza
| Challenge: | Experimental results show that a well-calibrated model is more reliable than a fine-tuned model due to “tuning-induced mis-calibration”. |
| Approach: | They propose a method which utilizes a small portion of teacher’s knowledge to obtain a reliable language model in a cost-efficient way. |
| Outcome: | The proposed method reduces the computational burden by utilizing teacher's knowledge to obtain a reliable language model in a cost-efficient way. |