Papers by Théo Dehaze
Understanding and Mitigating Language Confusion in LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Llama Instruct and Mistral models exhibit high degrees of language confusion and even the strongest models fail to consistently respond in the correct language. |
| Approach: | They develop a language confusion benchmark to evaluate LLMs' inability to consistently generate text in a user’s desired language. |
| Outcome: | The proposed model fails to consistently respond in the correct language, despite being prone to high temperatures and complex prompts. |