Papers by Sayontan Ghosh
PASTA: A Dataset for Modeling PArticipant STAtes in Narratives (2023.tacl-1)
Copied to clipboard
Sayontan Ghosh, Mahnaz Koupaee, Isabella Chen, Francis Ferraro, Nathanael Chambers, Niranjan Balasubramanian
| Challenge: | Existing models that understand narratives should infer these implicit states and their causal relationships with the narrative's explicit events. |
| Approach: | They propose a dataset that contains inferable participant states, a counterfactual perturbation to each state and the changes to the story that would be necessary if the counterfact was true. |
| Outcome: | The proposed model can reason about the impact of changes to the story that would be necessary if the counterfactual were true. |
Text-Derived Knowledge Helps Vision: A Simple Cross-modal Distillation for Video-based Action Anticipation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Prior work on action anticipation models treat it as a vision modality problem, but knowledge about action sequences can be obtained from textual data. |
| Approach: | They show how knowledge in pretrained language models can be adapted and distilled into vision based action anticipation models. |
| Outcome: | The proposed model achieves a 3.5% relative gain on EGTEA-GAZE+ and 7.2% relative gain for two action anticipation datasets. |
SpecNFS: A Challenge Dataset Towards Extracting Formal Models from Natural Language Specifications (2022.lrec-1)
Copied to clipboard
Sayontan Ghosh, Amanpreet Singh, Alex Merenstein, Wei Su, Scott A. Smolka, Erez Zadok, Niranjan Balasubramanian
| Challenge: | Existing methods for building formal semantic representations of specification texts are laborious and error-prone. |
| Approach: | They propose to use SpecIR to model sentences appearing in NFS specification documents as IF-THEN statements and introduce a representation language to parse them. |
| Outcome: | The proposed models achieve an F1 score of only 60.5 and 33.3 when using a state-of-the-art language model. |
SAGEViz: SchemA GEneration and Visualization (2023.emnlp-demo)
Copied to clipboard
Sugam Devare, Mahnaz Koupaee, Gautham Gunapati, Sayontan Ghosh, Sai Vallurupalli, Yash Kumar Lal, Francis Ferraro, Nathanael Chambers, Greg Durrett, Raymond Mooney, Katrin Erk, Niranjan Balasubramanian
| Challenge: | Schema induction involves creating a graph representation depicting how events unfold . supervised and few-shot approaches are not scalable and time-consuming . |
| Approach: | They propose a tool that utilizes human-AI collaboration to create and update complex schema graphs efficiently. |
| Outcome: | The proposed tool can generate schemas of better quality and be used by users in a variety of domains. |
POQue: Asking Participant-specific Outcome Questions for a Deeper Understanding of Complex Events (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing language models lag behind human performance in subtle ways in understanding complex situations, e.g., if the Argentine government yields to [IMF] pressure to rescind emergency legislation meant to protect ordinary families like the Brofmans. |
| Approach: | They propose to pre-identify a participant in a complex event and annotate their volitional engagement in causing the situation. |
| Outcome: | The proposed model can be used to infer the collective impact of salient events that make up a complex event, annotate volitional engagement of participants, and ground the outcome in state changes of the participants. |