| Challenge: | Relation Extraction (RE) evaluation is limited to in-domain setups . despite the drought of research on cross-domain RE, its practical importance remains . |
| Approach: | They propose a cross-domain benchmark for relation extraction which includes multi-label annotations and meta-data to include explanations and flags of difficult instances. |
| Outcome: | The proposed model includes explanations and flags of difficult instances. |
Similar Papers
What Do You Mean by Relation Extraction? A Survey on Datasets and Study on Scientific Relation Classification (2022.acl-srw)
Copied to clipboard
| Challenge: | Existing RE surveys focus on modeling techniques, but there are few that are based on real-world scenarios. |
| Approach: | They propose to survey RE datasets and revisit the task definition and its adoption by the community. |
| Outcome: | The proposed approach improves the reliability of RE evaluations across multiple datasets and reveals significant discrepancies in annotations. |
The State of Relation Extraction Data Quality: Is Bigger Always Better? (2024.findings-acl)
Copied to clipboard
| Challenge: | Relation extraction (RE) methods extract tuples of relationships from text . many datasets with frequent label errors have been used . |
| Approach: | They review recent surveys and a sample of recent RE methods papers . they find that real-time evaluations of RE methods are possible . |
| Outcome: | a sample of 38 datasets currently being used shows that many have frequent label errors . a small number of relations in specific domains can more realistically evaluate methods . |
CodRED: A Cross-Document Relation Extraction Dataset for Acquiring Knowledge in the Wild (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing relation extraction methods focus on extracting relational facts between entity pairs within single sentences or documents. |
| Approach: | They present a problem of cross-document relation extraction (CRE) using human annotations. |
| Outcome: | The proposed dataset is the first human-annotated cross-document RE dataset . it shows that it is challenging to existing RE methods including strong BERT-based models. |
AutoRE: Document-Level Relation Extraction with Large Language Models (2024.acl-demos)
Copied to clipboard
| Challenge: | Existing methods for relation extraction are limited to Sentence-level Relation Extraction (SentRE) tasks. |
| Approach: | They propose an end-to-end DocRE model that adopts a novel RE extraction paradigm named RHF (Relation-Head-Facts) Unlike existing approaches, AutoRE does not rely on the assumption of known relation options, making it more reflective of real-world scenarios. |
| Outcome: | The proposed model surpasses TAG by 10.03% and 9.03% on the dev and test set. |
More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction (2020.aacl-main)
Copied to clipboard
Xu Han, Tianyu Gao, Yankai Lin, Hao Peng, Yaoliang Yang, Chaojun Xiao, Zhiyuan Liu, Peng Li, Jie Zhou, Maosong Sun
| Challenge: | Existing methods for extracting relational facts from text have been successful . but with explosion of Web text, human knowledge is increasing drastically . |
| Approach: | They propose to improve relation extraction methods to extract relational facts from text . they analyze existing methods and show promising directions towards more powerful RE . |
| Outcome: | The proposed methods can extract relational facts from text, but they are still lacking in the current field. |
Entity-centered Cross-document Relation Extraction (2022.emnlp-main)
Copied to clipboard
Fengqi Wang, Fei Li, Hao Fei, Jingye Li, Shengqiong Wu, Fangfang Su, Wenxuan Shi, Donghong Ji, Bo Cai
| Challenge: | Existing methods for relation extraction only use text snippets surrounding target entities in multiple documents. |
| Approach: | They propose a relation-extraction model that uses cross-path entity relation attention to detect the semantic relations between entities in a given text. |
| Outcome: | The proposed method outperforms the state-of-the-art methods in the dataset CodRED by 10%. |
Let’s Stop Incorrect Comparisons in End-to-end Relation Extraction! (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing literature on Relation Extraction (RE) uses multiple evaluation setups to compare performance. |
| Approach: | They propose to quantify the most common comparison mistake and evaluate it leads to overestimating the final RE performance by around 5% on ACE05. |
| Outcome: | The proposed meta-analysis overestimates the final RE performance by around 5% on ACE05. |
Dialogue-Based Relation Extraction (2020.acl-main)
Copied to clipboard
| Challenge: | Existing dialogue-based relation extraction tasks focus on texts from formal genres such as professionally written and edited news reports or well-edited websites. |
| Approach: | They propose to use DialogRE to study cross-sentence relation extraction . they propose to annotate 36 possible relation types between arguments in dialogues . |
| Outcome: | The proposed dataset supports the prediction of relation(s) between two arguments that appear in a dialogue. |
Entity or Relation Embeddings? An Analysis of Encoding Strategies for Relation Extraction (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to relation extraction use concatenating embeddings of head and tail entities . however, such representations capture the types of the entities involved, leading to false positives and confusion between relations involving entities of the same type. |
| Approach: | They propose a model which combines [MASK] embeddings with entity embedds to learn relation embeddations. |
| Outcome: | The proposed model outperforms the state-of-the-art on several benchmarks . it uses a self-supervised pre-training strategy which further improves the results. |
Rethinking Document-Level Relation Extraction: A Reality Check (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent efforts push up performance boundaries of document-level relation extraction (DocRE) but these efforts are not promising. |
| Approach: | They construct four types of entity mention attacks to examine model robustness . they also have a close check on model usability in a more realistic setting . |
| Outcome: | The proposed model is based on a strong or untenable assumption in common . the model is robust under four types of mention attacks and usable in a realistic setting . |