Papers by Weilong Dong
Exploring Multilingual Concepts of Human Values in Large Language Models: Is Value Alignment Consistent, Transferable and Controllable across Languages? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Prior research has revealed that certain abstract concepts are linearly represented as directions in the representation space of LLMs, predominantly centered around English. |
| Approach: | They extend previous research that shows certain abstract concepts are linearly represented as directions in LLMs, predominantly centered around English. |
| Outcome: | The proposed model can be used to align LLMs with human values, and it can generate toxic, untruthful, biased, and even illegal content. |
CONTRANS: Weak-to-Strong Alignment Engineering via Concept Transplantation (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models behave consistently with human goals, values and intentions, but are computationally expensive. |
| Approach: | They propose a framework that enables weak-to-strong alignment transfer via concept transplantation. |
| Outcome: | The proposed framework surpasses instruction-tuned models in terms of truthfulness. |
DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have demonstrated that pretrained language models memorize and regurgitate a significant portion of training data, including atypical data points that appear only once in the training data. |
| Approach: | They propose a method to locate and erase risky neurons in order to eliminate the impact of privacy data in the model in batches. |
| Outcome: | The proposed method eliminates the impact of privacy data in the model in batches without affecting the model's performance. |
Mitigating Privacy Seesaw in Large Language Models: Augmented Privacy Neuron Editing via Activation Patching (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing privacy protection frameworks for large language models are inadequate . however, training data for LLMs often contains sensitive or unauthorized information . |
| Approach: | They propose a framework that augments privacy data by automatically synthesizing new private data, which deactivates the first trigger to privacy seesaw issue. |
| Outcome: | The proposed framework can mitigate Privacy Seesaw in large language models by synthesizing new private data and adapting activation patching to privacy neuron editing for switching off the second trigger. |