Papers by Eileen Wang
SCO-VIST: Social Interaction Commonsense Knowledge-based Visual Storytelling (2024.eacl-long)
Copied to clipboard
| Challenge: | Visual storytelling aims to automatically generate a coherent story based on a given image sequence. |
| Approach: | They propose a framework that represents the image sequence as a graph with objects and relations that includes human action motivation and its social interaction commonsense knowledge. |
| Outcome: | The proposed framework produces stories superior across multiple metrics in terms of visual grounding, coherence, diversity, and humanness, per both automatic and human evaluations. |
RoViST: Learning Robust Metrics for Visual Storytelling (2022.findings-naacl)
Copied to clipboard
| Challenge: | Visual storytelling is the task of generating a story paragraph that describes a given image sequence. |
| Approach: | They propose 3 evaluation metrics sets that analyze which aspects we would look for in a good story . they compare their correlation with human judgement scores on a sample of machine stories . |
| Outcome: | The proposed evaluation metrics outperform other metrics on human correlation on a sample of machine stories from state-of-the-art models. |