Papers by Blase Ur
Explaining Why: How Instructions and User Interfaces Impact Annotator Rationales When Labeling Text Data (2022.naacl-main)
Copied to clipboard
Jamar Sullivan Jr., Will Brackenbury, Andrew McNutt, Kevin Bryson, Kwam Byll, Yuxin Chen, Michael Littman, Chenhao Tan, Blase Ur
| Challenge: | In the context of data labeling, researchers are interested in having humans select rationales . |
| Approach: | They conducted an online user study to understand how humans select rationales . they found that participants were near unanimous in their data labels . |
| Outcome: | The results show that participants selected 12% of input tokens as rationales, but fewer if unable to drag over multiple tokens at once. |
Implicit Values Embedded in How Humans and LLMs Complete Subjective Everyday Tasks (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) can underpin AI assistants that help users with everyday tasks, such as making recommendations or performing basic computation. |
| Approach: | They audit how six popular large language models (LLMs) complete 30 everyday tasks and compare them to 100 human crowdworkers from the US. |
| Outcome: | The LLMs perform 30 tasks and are compared to 100 human crowdworkers in the US. |