Papers with Precision
INSIGHTBUDDY-AI: Medication Extraction and Entity Linking using Pre-Trained Language Models and Ensemble Learning (2025.naacl-srw)
Copied to clipboard
| Challenge: | InsightBuddy-AI is a system for extracting medication mentions and their associated attributes. |
| Approach: | They propose a system for extracting medication mentions and their associated attributes . they use stacked and voting ensembles built upon pre-trained language models . |
| Outcome: | The proposed system outperforms fine-tuned models in the extraction of medication mentions and associated attributes. |
Detecting Annotation Errors in Morphological Data with the Transformer (2022.acl-short)
Copied to clipboard
| Challenge: | Annotation errors that stem from various sources are usually unavoidable when performing large-scale annotation of linguistic data. |
| Approach: | They evaluate the feasibility of using a deep learning model to detect annotator errors in morphological data sets that contain inflected word forms. |
| Outcome: | The proposed model detects typographic errors, linguistic confusion errors and self-adversarial errors on four languages. |
Neural Retrieval for Question Answering with Cross-Attention Supervised Data Augmentation (2021.acl-short)
Copied to clipboard
| Challenge: | Early fusion models with cross-attention have shown better-than-human performance on some question answer benchmarks, while it is a poor fit for retrieval since it prevents pre-computation of the answer representations. |
| Approach: | They propose a supervised data mining method to train an efficient late fusion retrieval model by using cross-attention models with cross-references. |
| Outcome: | The proposed model outperforms retrieval models trained with gold annotations on Precision at N (P@N) and Mean Reciprocal Rank (MRR). |
PROBES : Performance and Relevance Observation for BEtter Search (2026.eacl-industry)
Copied to clipboard
| Challenge: | Qualitative search is essential for the success of online platforms, authors say . large-scale evaluation of search systems is essential to ensure high-quality user experiences . |
| Approach: | They propose a multi-task system powered by Large Language Models for end-to-end evaluation of semantic search systems. |
| Outcome: | The proposed system provides more precise and consistent relevance assessments across query categories. |
Balanced Accuracy: The Right Metric for Evaluating LLM Judges - Explained through Youden’s J statistic (2026.eacl-industry)
Copied to clipboard
| Challenge: | False refusals and task pass rates are key to reliable evaluation of large language models. |
| Approach: | They propose a principled best practice for evaluating judges based on a golden set of judge-quality metrics. |
| Outcome: | The proposed method improves the quality of judge-quality metrics on a golden set. |
Valency-Augmented Dependency Parsing (D18-1)
Copied to clipboard
| Challenge: | valency analysis is a complex task that requires a large number of subcategorizations, such as the number and types of syntactic dependents. |
| Approach: | They propose a parsing approach that explicitly models the number and types of syntactic dependents as valency patterns and a probabilistic model for tagging them. |
| Outcome: | The proposed approach outperforms the state-of-the-art labeled attachment score on 53 treebanks representing 41 languages and outperformed the previous state- of-the art labeles by 0.7. |
Reducing Quantity Hallucinations in Abstractive Summarization (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Abstractive summaries are subject to hallucination, but they are not very informative. |
| Approach: | They propose to use a beam-worth of abstractive summaries to up-rank summary that is not supported by the original text. |
| Outcome: | The proposed system up-ranks summaries whose quantity terms are supported by the original text without losing Recall, and shows higher Precision. |
Neural Relation Extraction via Inner-Sentence Noise Reduction and Transfer Learning (D18-1)
Copied to clipboard
| Challenge: | Existing methods for extracting relations are slow and lack precision . a novel approach to extract relations is proposed to reduce noise between sentences . |
| Approach: | They propose a word-level distant supervised approach for relation extraction using New York Times and Freebase. |
| Outcome: | The proposed method improves the area of precision/call(PR) from 0.35 to 0.39 over the state-of-the-art methods. |
uChecker: Masked Pretrained Language Models as Unsupervised Chinese Spelling Checkers (2022.coling-1)
Copied to clipboard
| Challenge: | Chinese Spelling Check (CSC) is a crucial and essential task in the area of natural language processing. |
| Approach: | They propose a framework to conduct unsupervised spelling error detection and correction using masked pretrained language models such as BERT. |
| Outcome: | The proposed model improves character-level and sentence-level accuracy, precision, recall, and F1-Measure on standard datasets. |
Tail-to-Tail Non-Autoregressive Sequence Prediction for Chinese Grammatical Error Correction (2021.acl-long)
Copied to clipboard
| Challenge: | Experimental results demonstrate the effectiveness of Tail-to-Tail (TtT) non-autoregressive sequence prediction for Chinese Grammatical Error Correction (CGEC) |
| Approach: | They propose a framework for Chinese Grammatical Error Correction (CGEC) that uses a BERT-initialized Transformer Encoder to model the error positions. |
| Outcome: | The proposed framework solves the problem of Chinese Grammatical Error Correction (CGEC) by modeling the token dependencies. |
Leveraging Linguistically Enhanced Embeddings for Open Information Extraction (2024.lrec-main)
Copied to clipboard
| Challenge: | Open Information Extraction (OIE) is a structure prediction task in NLP that aims to extract structured n-ary tuples from free text. |
| Approach: | They propose to leverage linguistic features with a Seq2Seq PLM for OIE to improve performance. |
| Outcome: | The proposed methods give any neural OIE architecture the key performance boost from both PLMs and linguistic features in one go. |
Beyond Correlation: Interpretable Evaluation of Machine Translation Metrics (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that MT metrics return assessments as scalar scores that are difficult to interpret, posing a challenge to making informed design choices. |
| Approach: | They propose an interpretable evaluation framework that evaluates MT metrics in two scenarios that serve as proxies for filtering and translation re-ranking use cases. |
| Outcome: | The proposed framework offers clearer insights than correlation with human judgments. |
Towards Better Value Principles for Large Language Model Alignment: A Systematic Evaluation and Enhancement (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) show remarkable performance across tasks . alignment with human values is critical for their responsible development. |
| Approach: | They propose a framework that evaluates value principles along three desirable properties . they propose supervised fine-tuning, reinforcement learning-based approaches . |
| Outcome: | The proposed framework improves value principles along the three desirable properties of LLMs. |