Papers by Daiqing Qi
Tag-grounded Visual Instruction Tuning with Retrieval Augmentation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) have seen remarkable progress in providing general instruction-following ability, but struggle with critical problems when required to provide a detailed and accurate response to a visual instruction. |
| Approach: | They propose to enhance the mapping process by using retrieval-augmented tag tokens, which contain rich object-aware information such as object names and attributes. |
| Outcome: | The proposed model outperforms baselines that share the same language model and training data on 12 benchmarks and shows zero-shot capability when provided with specific datastores. |
No Free Lunch: Retrieval-Augmented Generation Undermines Fairness in LLMs, Even for Vigilant Users (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Retrieval-augmented generation is widely adopted for its effectiveness and cost-efficiency in mitigating hallucinations. |
| Approach: | They propose a practical three-level threat model from the perspective of user fairness awareness. |
| Outcome: | The proposed model shows that RAG can undermine fairness alignment without fine-tuning or retraining. |