Introducing a New Dataset for Event Detection in Cybersecurity Texts (2020.emnlp-main)
Copied to clipboard
| Challenge: | a large amount of text data is produced to report and discuss cyber vulnerabilities . detecting cybersecurity events is necessary to keep us informed about the fast growing number of such events reported in text. |
| Approach: | They propose a dataset characterizing the manual annotation for 30 important cybersecurity event types and a large dataset to develop deep learning models. |
| Outcome: | The proposed dataset characterizes the manual annotation for 30 important event types and supports the modeling of document-level information to improve the performance. |
Similar Papers
MAVEN: A Massive General Domain Event Detection Dataset (2020.emnlp-main)
Copied to clipboard
Xiaozhi Wang, Ziqi Wang, Xu Han, Wangyi Jiang, Rong Han, Zhiyuan Liu, Juanzi Li, Peng Li, Yankai Lin, Jie Zhou
| Challenge: | Existing datasets exhibit data scarcity and limited coverage of general-domain events. |
| Approach: | They present a MAssive eVENt detection dataset which contains 4,480 Wikipedia documents and 168 event types. |
| Outcome: | The proposed dataset shows that existing methods cannot achieve promising results on the small datasets. |
MUSIED: A Benchmark for Event Detection from Multi-Source Heterogeneous Informal Texts (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent efforts to classify unstructured texts into specific types have been limited in practical scenarios. |
| Approach: | They propose to use Chinese text conversations and phone conversations to expand event detection to the scenarios involving informal and heterogeneous texts. |
| Outcome: | The proposed dataset is based on user reviews, text conversations, and phone conversations in a leading e-commerce platform for food service. |
MINION: a Large-Scale and Diverse Dataset for Multilingual Event Detection (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for ED in IE and NLP focus on feature-based models to feature-driven models. |
| Approach: | They propose to use a multilingual dataset to annotate events for 8 different languages . they demonstrate the challenges and transferability of ED across languages in MINION . |
| Outcome: | a new dataset that consistently annotates events for 8 different languages is released . the new dataset will promote future research on multilingual ED . |
Learning with Partial Annotations for Event Detection (2023.acl-long)
Copied to clipboard
| Challenge: | Event detection (ED) requires fully labeled and high-quality training data. |
| Approach: | They propose a new trigger localization formulation using contrastive learning to distinguish ground-truth triggers from contexts and show a decent robustness for addressing partial annotation noise. |
| Outcome: | The proposed approach achieves an F1 score of over 60% in an extreme scenario where 90% of events are unlabeled. |
Literary Event Detection (P19-1)
Copied to clipboard
| Challenge: | a new dataset of literary events is presented to examine the nature of narratives . literature presents a number of challenges for existing systems, including complex narration . |
| Approach: | They propose a dataset of literary events that are depicted as taking place within the imagined space of a novel. |
| Outcome: | The proposed model achieves an F1 score of 73.9 for prestige and popularity . the best performing model achieve a score of 79.9 for prestige compared to the previous model . |
Forecasting Future International Events: A Reliable Dataset for Text-Based Event Modeling (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches for text-based event prediction are limited in quality due to dynamic nature of international relations and conflicting economic dynamics. |
| Approach: | They propose a novel dataset that leverages the advanced reasoning capabilities of large-language models to address these limitations. |
| Outcome: | The proposed dataset features high-quality scoring labels generated through advanced prompt modeling and rigorously validated by domain experts in political science. |
DEIE: Benchmarking Document-level Event Information Extraction with a Large-scale Chinese News Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing event-based datasets mainly target sentence-level tasks . current models struggle with "document" annotation, a key feature of the current model . |
| Approach: | They propose a large-scale document-level event information extraction dataset with over 56,000+ events and 242,000+ arguments. |
| Outcome: | The proposed dataset has over 56,000+ events and 242,000+ arguments. |
AnnoCTR: A Dataset for Detecting and Linking Entities, Tactics, and Techniques in Cyber Threat Reports (2024.lrec-main)
Copied to clipboard
Lukas Lange, Marc Müller, Ghazaleh Haratinezhad Torbati, Dragan Milchevski, Patrick Grau, Subhash Chandra Pujari, Annemarie Friedrich
| Challenge: | Abstract: Natural language processing can help with managing large amounts of unstructured information. |
| Approach: | They propose to annotate a CC-BY-SA-licensed dataset of cyber threat reports . they use named entities, temporal expressions, and cybersecurity-specific concepts . |
| Outcome: | The proposed dataset annotates reports with named entities, temporal expressions, and cybersecurity-specific concepts including implicitly mentioned techniques and tactics. |
Fine-Grained Event Trigger Detection (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for Event Detection (ED) focus on a limited set of event types . existing datasets for ED focus on only 33 event types while the number of events in the TAC KBP dataset is 38. |
| Approach: | They propose to transform Word Sense Disambiguation's Semcor dataset into a large and high-quality dataset for ED. |
| Outcome: | The proposed method transforms the Semcor dataset for Word Sense Disambiguation into a large and high-quality dataset for FED. |
Treasures Outside Contexts: Improving Event Detection via Global Statistics (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing neural-based ED models are confused by changeable contexts during testing . we propose a system that extracts statistical event features from word-event cooccurrence frequencies . |
| Approach: | They propose to integrate a set of statistical event features from word-event co-occurrence frequencies into the training set to cooperate with contextual features. |
| Outcome: | The proposed model outperforms ten strong baselines on ACE2005 and KBP2015 datasets. |