Papers by Shihao Ran
Event Extraction as Question Generation and Answering (2023.acl-short)
Copied to clipboard
| Challenge: | Recent work on Event Extraction addresses the error propagation issue found in token-based classification approaches. |
| Approach: | They propose a Question Generation (QG) model that generates questions that leverage contextual information instead of fixed templates. |
| Outcome: | The proposed model outperforms all previous single-task-based models on the ACE05 English dataset. |
BUMP: A Benchmark of Unfaithful Minimal Pairs for Meta-Evaluation of Faithfulness Metrics (2023.acl-long)
Copied to clipboard
Liang Ma, Shuyang Cao, Robert L Logan IV, Di Lu, Shihao Ran, Ke Zhang, Joel Tetreault, Alejandro Jaimes
| Challenge: | Existing benchmarks measure the correlation with human judgements of faithfulness on model-generated summaries, but they are insufficient for diagnosing whether metrics are consistent, effective on human-written texts, and sensitive to different error types. |
| Approach: | They propose to use unfaithful minimal pairs to measure the consistency of automatic faithfulness metrics by comparing human-written summary pairs with a dataset of 889 human-writing, minimally different summary pairs. |
| Outcome: | The proposed benchmarks show that the most discriminative metrics tend not to be the most consistent, and that the best performing metrics are sensitive to errors. |
A New Task and Dataset on Detecting Attacks on Human Rights Defenders (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets for event extraction cannot extract information from textual sources. |
| Approach: | They propose to use crowdsourced annotations on 500 online news articles to train and evaluate baseline models to predict annotated characteristics. |
| Outcome: | The proposed dataset includes crowdsourced annotations on 500 online news articles and includes fine-grained information about the type and location of attacks, as well as information about victims. |
CrisisLTLSum: A Benchmark for Local Crisis Event Timeline Extraction and Summarization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Timeline extraction and abstractive summarization are critical tasks for leveraging large numbers of social media posts about events. |
| Approach: | They propose to build a semi-automated cluster-then-refine algorithm to extract local crisis event timelines from Twitter. |
| Outcome: | The proposed approach performs better than human models on extraction and summarization tasks. |
CEHA: A Dataset of Conflict Events in the Horn of Africa (2025.coling-main)
Copied to clipboard
Rui Bai, Di Lu, Shihao Ran, Elizabeth M. Olson, Hemank Lamba, Aoife Cahill, Joel Tetreault, Alejandro Jaimes
| Challenge: | Existing datasets categorizing conflict events do not cover all of the fine-grained types of conflict relevant to areas like the Horn of Africa. |
| Approach: | They propose to use online news articles to categorize violent conflict events . they propose to extract event-relevance and event-types from 500 English event descriptions . |
| Outcome: | The proposed dataset categorizes conflict risk according to specific areas required by stakeholders in the Humanitarian-Peace-Development Nexus. |