Papers with MemoTrap
DeCoRe: Decoding by Contrasting Retrieval Heads to Mitigate Hallucinations (2025.findings-emnlp)
Copied to clipboard
Aryo Pradipta Gema, Chen Jin, Ahmed Abdulaal, Tom Diethe, Philip Alexander Teare, Beatrice Alex, Pasquale Minervini, Amrutha Saseendran
| Challenge: | Large Language Models often produce unfaithful or factually incorrect outputs . masking retrieval heads can induce hallucinations, but decoding by contrast can reduce hallucinosity . |
| Approach: | They propose a training-free decoding strategy that contrasts the outputs of the base LLM and the masked LLM. |
| Outcome: | The proposed decoding strategy reduces hallucinations by contrasting the outputs of the base and masked LLMs. |
The Law of Knowledge Overshadowing: Towards Understanding, Predicting and Preventing LLM Hallucination (2025.findings-acl)
Copied to clipboard
Yuji Zhang, Sha Li, Cheng Qian, Jiateng Liu, Pengfei Yu, Chi Han, Yi R. Fung, Kathleen McKeown, ChengXiang Zhai, Manling Li, Heng Ji
| Challenge: | Hallucination is a persistent challenge in large language models where even with rigorous quality control, models often generate distorted facts. |
| Approach: | They propose a new framework to quantify factual hallucinations by modeling knowledge overshadowing. |
| Outcome: | The proposed framework improves model factuality on Overshadow (27.9%), MemoTrap (13.1%) and NQ-Swap (18.3%). |