Papers by Jessica Echterhoff
MUSCLE: A Model Update Strategy for Compatible LLM Evolution (2024.findings-emnlp)
Copied to clipboard
Jessica Echterhoff, Fartash Faghri, Raviteja Vemulapalli, Ting-Yao Hu, Chun-Liang Li, Oncel Tuzel, Hadi Pouransari
| Challenge: | Large Language Models (LLMs) are regularly updated to enhance performance, typically through changes in data or architecture. |
| Approach: | They propose a method to minimize the extent of instance regression in model updates by training a compatibility adapter that can enhance task fine-tuned language models. |
| Outcome: | The proposed approach reduces regressions by up to 40% when updating LLMs to newer versions while maintaining overall performance gains. |
Cognitive Bias in Decision-Making with LLMs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models inherit societal biases against protected groups and can be subject to functionally resembling cognitive bias. |
| Approach: | They propose a framework to uncover, evaluate, and mitigate cognitive bias in large language models by using a dataset containing 13,465 prompts to evaluate LLM decisions on different cognitive biases. |
| Outcome: | The proposed framework uncovers, evaluates, and mitigates cognitive bias in large language models, particularly in high-stakes decision-making tasks. |
Evaluating Large Language Models as Generative User Simulators for Conversational Recommendation (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models show promise in simulating human-like behavior, raising the question of their ability to represent a diverse population of users. |
| Approach: | They propose a protocol to evaluate the degree to which language models can accurately emulate human behavior in conversational recommendation systems. |
| Outcome: | The proposed protocol evaluates five tasks to reveal deviations of language models from human behavior and offers insights on how to reduce deviations with model selection and prompting strategies. |