Papers by Kory Mathewson
Evaluating Coherence in Dialogue Systems using Entailment (N19-1)
Copied to clipboard
| Challenge: | Evaluating open-domain dialogue systems is difficult due to the diversity of possible correct answers. |
| Approach: | They propose a set of metrics for evaluating topic coherence using distributed sentence representations and calculable approximations of human judgment using conversational coherency. |
| Outcome: | The proposed metrics can be used as a surrogate for human judgment based on conversational coherence on large-scale datasets and provide an unbiased estimate for the quality of the responses. |
Story Centaur: Large Language Model Few Shot Learning as a Creative Writing Tool (2021.eacl-demos)
Copied to clipboard
| Challenge: | Few shot learning with large language models has the potential to give individuals without formal machine learning training access to a wide range of text to text models. |
| Approach: | They propose a user interface for prototyping few shot models and a set of recombinable web components that deploy them. |
| Outcome: | The proposed interface lets writers build their own co-creation tools that further their own artistic directions. |
Can language models learn from explanations in context? (2022.findings-emnlp)
Copied to clipboard
Andrew Lampinen, Ishita Dasgupta, Stephanie Chan, Kory Mathewson, Mh Tessler, Antonia Creswell, James McClelland, Jane Wang, Felix Hill
| Challenge: | Language Models can adapt to a few in-context examples, but without training. |
| Approach: | They examine how explanations of few-shot examples can help Language Models (LMs) explanations can improve performance even without tuning, they find . |
| Outcome: | The proposed explanations outperform hand-tuned explanations on small validation sets. |