Papers with token-flipping
Improving Conversational Abilities of Quantized Large Language Models via Direct Preference Alignment (2024.acl-long)
Copied to clipboard
| Challenge: | Quantization-aware direct preference optimization (QDPO) improves conversational abilities of quantized LLMs . token-flipping is a critical factor for degraded text generation quality . |
| Approach: | They propose a method that aligns quantized LLMs with their full-precision counterparts, improving conversational abilities. |
| Outcome: | The proposed method outperforms established fine-tuning techniques on two instruction-tuned LLMs in various languages and models, setting a new benchmark for conversational chatbot development. |