Papers by Alex Chandler
On Measuring Social Biases in Sentence Encoders (N19-1)
Copied to clipboard
| Challenge: | Word embeddings such as word2vec and GloVe exhibit human-like implicit biases based on gender, race, and other social constructs. |
| Approach: | They propose a simple generaliza test to measure bias in word embeddings by comparing two sets of target-concept words to two sets . |
| Outcome: | The proposed test shows that word2vec and word2Ve exhibit human-like implicit biases based on gender, race, and other social constructs. |
Evaluating Language Model Character Traits (2024.findings-emnlp)
Copied to clipboard
Francis Ward, Zejia Yang, Alex Jackson, Randy Brown, Chandler Smith, Grace Colverd, Louis Thomson, Raymond Douglas, Patrik Bartak, Andrew Rowan
| Challenge: | Language models (LMs) exhibit human-like behaviour, but it is unclear how to describe this behaviour without undue anthropomorphism. |
| Approach: | They formalise a behaviourist view of LM character traits and infer belief and intent from LM behaviour, finding consistency varies with model size, fine-tuning, and prompting. |
| Outcome: | The proposed model enables us to describe LM behaviour precisely and without undue anthropomorphism. |
Detecting Errors through Ensembling Prompts (DEEP): An End-to-End LLM Framework for Detecting Factual Errors (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for detecting factual errors in text summarization are inadequate for the task. |
| Approach: | They propose an end-to-end large language model framework for detecting factual errors in text summarization. |
| Outcome: | The proposed framework achieves state-of-the-art (SOTA) balanced accuracy on the AggreFact-XSUM FTSOTA, TofuEval Summary-Level, and HaluEVAL Summarization benchmarks. |