Papers by Xiangyu Hong
On Large Language Models’ Hallucination with Regard to Known Facts (2024.naacl-long)
Copied to clipboard
Che Jiang, Biqing Qi, Xiangyu Hong, Dayuan Fu, Yang Cheng, Fandong Meng, Mo Yu, Bowen Zhou, Jie Zhou
| Challenge: | Large language models are successful in answering factoid questions but are also prone to hallucination. |
| Approach: | They propose self-reporting to the model when faced with such limitations. |
| Outcome: | The proposed classifier can detect hallucinations with an 88% success rate and can be used to answer factoid questions with correct answer knowledge. |
On the token distance modeling ability of higher RoPE attention dimension (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on extending the context length of language models based on Rotary position embedding (RoPE) has shown promising results in capturing longer-range contextual information. |
| Approach: | They propose to use a hidden dimension of an attention head to investigate its contribution to capturing long-distance dependencies. |
| Outcome: | The proposed model can capture long-distance dependencies by extending the attention of a particular dimension of an attention head. |