P-QuASAR: A Unified Probabilistic Framework for Holistic Patent Quality Assessment and Refinement (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for assessing patent quality rely on modular pipelines or generic detectors, resulting in fragmented decisions and limited integration across quality dimensions. |
| Approach: | They propose a probabilistic framework that represents patent specifications as Quality Graphs. |
| Outcome: | The proposed framework outperforms existing methods on 500 patents against seven baselines. |
Similar Papers
PatentMind: A Multi-Aspect Reasoning Graph for Patent Similarity Evaluation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for patent similarity evaluation lack the multifaceted structure of patent documents . patent documents pose significant challenges due to specialized domain knowledge, intricate legal language, and complex structural formats. |
| Approach: | They propose a framework that performs patent similarity evaluation through a Multi-Aspect Reasoning Graph. |
| Outcome: | The proposed framework outperforms embedding-based, patent-specific, and prompt engineering benchmarks in evaluating patent similarity with expert annotations. |
T5Score: A Methodology for Automatically Assessing the Quality of LLM Generated Multi-Document Topic Sets (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation methods for Multi-Document Topic Extraction are not designed for LLMs and result in low inter-annotator agreement scores. |
| Approach: | They propose an evaluation methodology that decomposes the quality of a topic set into quantifiable aspects, measurable through easy-to-perform annotation tasks. |
| Outcome: | The proposed evaluation methodology decomposes the quality of a topic set into quantifiable aspects, measurable through easy-to-perform annotation tasks. |
ReportLogic: Evaluating Logical Quality in Deep Research Reports (2026.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation frameworks that evaluate large language models for Deep Research largely ignore this requirement. |
| Approach: | They propose a benchmark that quantifies report-level logical quality through a reader-centric lens of auditability. |
| Outcome: | The proposed model quantifies logical quality through a reader-centric lens of auditability. |
Probabilistic Soundness Guarantees in LLM Reasoning Chains (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for detecting propagated errors in reasoning chains are inadequate . author et al. (2017) show that initial errors propagate and undermine reliability of final conclusion . |
| Approach: | They propose a framework that evaluates each reasoning step based solely on previously-verified premises and provides certified statistical guarantees of its soundness. |
| Outcome: | ARES achieves state-of-the-art performance across four benchmarks and demonstrates superior robustness on very long synthetic reasoning chains. |
Patentformer: A Novel Method to Automate the Generation of Patent Applications (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Patentformer is a novel method for generating patent specification by fine-tuning the generative models with diverse sources of information, e.g., patent claims, drawing text, and brief descriptions of the drawings. |
| Approach: | They propose a method for generating patent specification by fine-tuning generative models with diverse sources of information, e.g., patent claims, drawing text, and brief descriptions of the drawings. |
| Outcome: | The proposed method generates patent specification in legal writing style and human-like quality may be better than the actual specification. |
PatentScore: Multi-dimensional Evaluation of LLM-Generated Patent Claims (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing natural language generation (NLG) metrics fail to capture domain-specific nuances . patent claims require precise assessment of structural elements such as antecedent consistency and claim dependency. |
| Approach: | They propose a multi-dimensional evaluation framework specifically designed for patent claims . PatentScore integrates hierarchical decomposition of claim elements, validation patterns and scoring across structural, semantic, and legal dimensions. |
| Outcome: | The proposed evaluation framework outperforms existing evaluation frameworks on patent claims . patentScore achieved highest correlation with expert annotations on 400 patent claims dataset . |
Quality Assessment of Tabular Data using Large Language Models and Code Generation (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Data quality is vital for business decisions; poor data quality costs organizations an average of $12.9 million annually. |
| Approach: | They propose a framework that combines statistical inliner detection with LLM-driven rule and code generation. |
| Outcome: | The proposed framework produces semantically valid quality rules and validates them with retrieval-augmented generation (RAG) Extensive evaluations on benchmark datasets confirm the effectiveness of the proposed framework. |
Towards Better Evaluation for Generated Patent Claims (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies highlight inconsistencies between automated evaluation metrics and human expert assessments for patent claims. |
| Approach: | They propose a multi-dimensional evaluation method specifically designed for patent claims that incorporates features annotated by patent experts. |
| Outcome: | The proposed method achieves highest correlation with human expert evaluations across all assessment criteria across all tested metrics. |
LOGICAL-COMMONSENSEQA: A Benchmark for Logical Commonsense Reasoning (2026.acl-short)
Copied to clipboard
| Challenge: | LOGICAL-COMMONSENSEQA benchmarks evaluate commonsense reasoning as logical composition over pairs of atomic statements . commonsensible reasoning is central to human cognition and a long-standing challenge in artificial intelligence and natural language understanding. |
| Approach: | They propose a benchmark that reframes commonsense reasoning as logical composition over pairs of atomic statements using plausibility-level operators. |
| Outcome: | LOGICAL-COMMONSENSEQA exposes fundamental reasoning limitations and provides a framework for advancing compositional commonsense reasoning. |
Automatic Reviewers Fail to Detect Faulty Reasoning in Research Papers: A New Counterfactual Evaluation Framework (2026.tacl-1)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used as fully automatic review generators (ARGs). |
| Approach: | They propose a fully automated counterfactual evaluation framework that isolates and tests a core review skill that underpins high-quality peer review: detecting faulty research logic. |
| Outcome: | The proposed framework isolates and tests a range of ARG approaches and shows that flaws in research logic have no significant effect on their output reviews. |