Papers with Llama3.2
Forewarned is Forearmed: Pre-Synthesizing Jailbreak-like Instructions to Enhance LLM Safety Guardrail to Potential Attacks (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Despite advances in improving large language model (LLM) to refuse to answer malicious instructions, LLMs remain vulnerable to jailbreak attacks where attackers generate instructions with distributions differing from safety alignment corpora. |
| Approach: | They propose a framework that leverages embedding space distribution analysis to generate jailbreak-like instructions. |
| Outcome: | The proposed framework shows significant decreases in attack success rate on Qwen2.5, Llama3.1, and Llma3.2 without compromising their utility. |
Multilingual Language Model Pretraining using Machine-translated Data (2025.emnlp-main)
Copied to clipboard
Jiayi Wang, Yao Lu, Maurice Weber, Max Ryabinin, David Ifeoluwa Adelani, Yihong Chen, Raphael Tang, Pontus Stenetorp
| Challenge: | Existing methods for collecting and filtering multilingual web data lead to most languages lagging behind English performance due to the Internet's English-centric nature. |
| Approach: | They propose to translate a high-quality English web corpus into nine languages and pretrain a 1.3B-parameter model on it. |
| Outcome: | The proposed model matches or outperforms multilingual LLMs of similar size across Non-English understanding and reasoning tasks despite being trained on an order of magnitude less data. |