Linfeng Song, Ante Wang, Xiaoman Pan, Hongming Zhang, Dian Yu, Lifeng Jin, Haitao Mi, Jinsong Su, Yue Zhang, Dong Yu
| Challenge: | Existing OIE systems organize knowledge into subject-relation-object (SRO) triplets, and they use templates to extract such knowledge triplet. |
| Approach: | They propose a framework to handle expressiveness and groundedness in OpenFact . they propose to use templates, extra constraints, and adopt human efforts to ensure that most triplets contain enough details. |
| Outcome: | The proposed framework improves expressiveness and groundedness of OpenFact . it is more accurate and denser than OPIEC-Linked, which is grounded to Wikidata . |
Similar Papers
ComFact: A Benchmark for Linking Contextual Commonsense Knowledge (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to retrieve facts from commonsense knowledge graphs are imprecise, requiring heuristics that ignore contexts and ambiguity . a novel benchmark, ComFact, contains 293k in-context relevance annotations for commonsensense triplets . |
| Approach: | They propose a task of commonsense fact linking where models are given contexts and trained to identify situationally-relevant commonsensical knowledge from KGs. |
| Outcome: | The proposed benchmark shows that heuristic fact linking approaches are imprecise . however, the models still significantly underperform humans in the commonsense augmentation task . |
VeriFact: Enhancing Long-Form Factuality Evaluation with Refined Fact Extraction and Reference Facts (2025.emnlp-main)
Copied to clipboard
| Challenge: | Prior work focuses on accuracy and precision, but factuality evaluation is difficult due to inter-sentence dependencies. |
| Approach: | They introduce a factuality evaluation framework to enhance fact extraction . they also introduce 'factRBench' that evaluates both precision and recall . |
| Outcome: | The proposed framework enhances fact extraction by identifying incomplete and missing facts . it also evaluates precision and recall in long-form models, whereas prior work focuses on precision. |
OpenFactCheck: A Unified Framework for Factuality Evaluation of LLMs (2024.emnlp-demo)
Copied to clipboard
| Challenge: | Large language models (LLMs) often produce content that deviates from real-world facts. |
| Approach: | They developed a unified framework to assess the factuality of large language models . open-sourced framework is publicly available as a Python library and web service . |
| Outcome: | OpenFactCheck is open-sourced and publicly released as a Python library and web service. |
OpenKI: Integrating Open Information Extraction and Knowledge Bases with Relation Inference (N19-1)
Copied to clipboard
| Challenge: | Existing methods for knowledge extraction and alignment are limited in quality and performance. |
| Approach: | They propose to integrate OpenIE extractions in the form of (subject, predicate, object) triples with Knowledge Bases (KB) |
| Outcome: | The proposed method improves state-of-the-art for OpenIE extractions and boosts performance on OpenIE from semi-structured data. |
Linking Surface Facts to Large-Scale Knowledge Graphs (2023.emnlp-main)
Copied to clipboard
| Challenge: | Open Information Extraction (OIE) methods extract facts in the form of triples . ambiguity of these triples hinders their downstream usage . |
| Approach: | They propose a benchmark that measures fact linking performance on a granular triple slot level . they propose to use a system that can detect out-of-KG entities and predicates . |
| Outcome: | The proposed benchmark can measure fact linking performance on a granular triple slot level while also measuring if a system can recognize that a surface form has no match in the existing KG. |
OVFact: Measuring and Improving Open-Vocabulary Factuality for Long Caption Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large vision-language models struggle to generate long and factual captions . traditional measures for hallucination and factuality are not well suited for longer captions. |
| Approach: | They propose a method for measuring caption factuality of long captions that leverages open-vocabulary visual grounding and tool-based verification without relying on human annotations. |
| Outcome: | The proposed method improves agreement with human judgements and captures both caption descriptiveness and factual precision in the same metric. |
OpenFactCheck: Building, Benchmarking Customized Fact-Checking Systems and Evaluating the Factuality of Claims and LLMs (2025.coling-main)
Copied to clipboard
Yuxia Wang, Minghan Wang, Hasan Iqbal, Georgi N. Georgiev, Jiahui Geng, Iryna Gurevych, Preslav Nakov
| Challenge: | Large language models (LLMs) generate naturallysounding answers over a broad range of human inquiries, but they still produce content that deviates from real-world facts. |
| Approach: | They propose a framework for building customized automatic fact-checking systems, benchmarking their accuracy, evaluating factuality of LLMs, and verifying claims in a document. |
| Outcome: | The proposed framework assesses the factuality of free-form responses in open domains and evaluates factually of LLMs. |
BenchIE: A Framework for Multi-Faceted Fact-Based Open Information Extraction Evaluation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for OIE are incomplete and do not include all acceptable variants of the same fact. |
| Approach: | They introduce BenchIE: a benchmark and evaluation framework for comprehensive evaluation of OIE systems for English, Chinese, and German. |
| Outcome: | The proposed framework is based on fact synsets, clusters, and standardized benchmarks. |
Grounded Keys-to-Text Generation: Towards Factual Open-Ended Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Large pre-trained language models have enabled open-ended generation frameworks to tackle a variety of tasks beyond data-to-text generation. |
| Approach: | They propose a new task to generate a factual description about an entity given guiding keys and grounding passages using a dataset. |
| Outcome: | The proposed model improves factual correctness and recall significantly compared to previous models. |
WebIE: Faithful and Robust Information Extraction on the Web (2023.acl-long)
Copied to clipboard
| Challenge: | Existing closed IE datasets are built using Wikipedia, but they have limitations when applied to web domains. |
| Approach: | They propose to annotate 25K triples from WebIE through crowdsourcing and introduce mWebIE, a translation of the annotated set in four other languages. |
| Outcome: | The proposed model trains on 1.6M sentences from the English Common Crawl corpus and includes negative examples to better reflect the data on the web. |