Papers by Michael Littman
Explaining Why: How Instructions and User Interfaces Impact Annotator Rationales When Labeling Text Data (2022.naacl-main)
Copied to clipboard
Jamar Sullivan Jr., Will Brackenbury, Andrew McNutt, Kevin Bryson, Kwam Byll, Yuxin Chen, Michael Littman, Chenhao Tan, Blase Ur
| Challenge: | In the context of data labeling, researchers are interested in having humans select rationales . |
| Approach: | They conducted an online user study to understand how humans select rationales . they found that participants were near unanimous in their data labels . |
| Outcome: | The results show that participants selected 12% of input tokens as rationales, but fewer if unable to drag over multiple tokens at once. |
Planetarium: A Rigorous Benchmark for Translating Text to Structured Planning Languages (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing evaluation methods struggle to ensure semantic correctness and rely on simple or unrealistic datasets. |
| Approach: | They propose a benchmark to evaluate language models’ ability to generate PDDL code from natural language descriptions of planning tasks. |
| Outcome: | The proposed benchmark evaluates the ability of language models to generate PDDL code from natural language descriptions of planning tasks against ground truth and a dataset of 145,918 text-to-PDDL pairs with varying levels of difficulty. |