Papers by Raymond Douglas
Evaluating Language Model Character Traits (2024.findings-emnlp)
Copied to clipboard
Francis Ward, Zejia Yang, Alex Jackson, Randy Brown, Chandler Smith, Grace Colverd, Louis Thomson, Raymond Douglas, Patrik Bartak, Andrew Rowan
| Challenge: | Language models (LMs) exhibit human-like behaviour, but it is unclear how to describe this behaviour without undue anthropomorphism. |
| Approach: | They formalise a behaviourist view of LM character traits and infer belief and intent from LM behaviour, finding consistency varies with model size, fine-tuning, and prompting. |
| Outcome: | The proposed model enables us to describe LM behaviour precisely and without undue anthropomorphism. |