Papers by Erica Cai
The State of Relation Extraction Data Quality: Is Bigger Always Better? (2024.findings-acl)
Copied to clipboard
| Challenge: | Relation extraction (RE) methods extract tuples of relationships from text . many datasets with frequent label errors have been used . |
| Approach: | They review recent surveys and a sample of recent RE methods papers . they find that real-time evaluations of RE methods are possible . |
| Outcome: | a sample of 38 datasets currently being used shows that many have frequent label errors . a small number of relations in specific domains can more realistically evaluate methods . |
Evaluating Zero-Shot Event Structures: Recommendations for Automatic Content Extraction (ACE) Annotations (2023.acl-short)
Copied to clipboard
| Challenge: | Zero-shot event extraction (EE) methods infer richly structured event records from unstructured text data, based on a user-supplied natural language specification and no training examples. |
| Approach: | They propose recommendations for future evaluations so the research community can better utilize ACE as an event evaluation resource. |
| Outcome: | The proposed methods can be used to evaluate zero-shot and other low-supervision EE methods, considering up to 32% of correctly identified arguments and 25% of correctly ignored event mentions as false negatives. |