Papers by Juraj Vladika
Facts Fade Fast: Evaluating Memorization of Outdated Medical Knowledge in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | LLMs encode extensive knowledge within their parameters, but the knowledge in LLM models can become outdated over time. |
| Approach: | They propose two new LLMs that provide outdated medical advice . they compare the models with a set of QA pairs whose verdict changed through time . |
| Outcome: | The proposed models exhibit memorization of outdated knowledge to some extent. |
Comparing Knowledge Sources for Open-Domain Scientific Claim Verification (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing systems for fact-checking scientific claims assume that the documents containing the evidence are already provided and annotated or contained in a limited corpus. |
| Approach: | They perform an array of experiments to test the performance of open-domain claim verification systems on four datasets of biomedical and health claims in different settings. |
| Outcome: | The proposed system performs better with biomedical and health claims, while Wikipedia is more suited for everyday health concerns. |
A Decade of Knowledge Graphs in Natural Language Processing: A Survey (2022.aacl-main)
Copied to clipboard
| Challenge: | Knowledge graphs (KGs) are a representation of semantic relations between entities . despite their popularity, there is still no general understanding of what exactly a KG is or for what tasks it is applicable. |
| Approach: | They analyze 507 papers on knowledge graphs in natural language processing (NLP) they provide a taxonomy of tasks and review the maturity of individual research streams . |
| Outcome: | The findings summarize the literature and highlight directions for future work. |
MedREQAL: Examining Medical Knowledge Recall of Large Language Models via Question Answering (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models can encode knowledge during pre-training on large text corpora, enabling downstream tasks like question answering (QA). |
| Approach: | They construct a dataset derived from systematic reviews to examine their ability to encode medical knowledge and their recall. |
| Outcome: | The proposed model performs well on the biomedical QA dataset. |
On the Influence of Context Size and Model Choice in Retrieval-Augmented Generation Systems (2025.findings-naacl)
Copied to clipboard
| Challenge: | Retrieval-augmented generation (RAG) is an approach to augment large language models (LLMs) despite their impressive performance, LLMs can generate plausible sounding but factually incorrect responses (hallucinations) |
| Approach: | They propose to use BM25 and semantic search as retrievers to augment large language models by reducing their reliance on static knowledge and improving answer factuality. |
| Outcome: | The proposed approach improves QA performance on a biomedical task with up to 15 snippets but stagnates or declines beyond that. |
Improving Health Question Answering with Reliable and Time-Aware Evidence Retrieval (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing question answering systems rely on pre-selected and annotated evidence documents, thus making them inadequate for addressing novel questions. |
| Approach: | They propose to use the common retrieve-then-read QA pipeline and PubMed as a trustworthy collection of medical research documents to answer health questions from three diverse datasets. |
| Outcome: | The proposed approach improves the macro F1 score by 10% by utilizing the common retrieve-then-read QA pipeline and PubMed as a trustworthy collection of medical research documents. |
HealthFC: Verifying Health Claims with Evidence-Based Medical Fact-Checking (2024.lrec-main)
Copied to clipboard
| Challenge: | determining the trustworthiness of online medical content is challenging in the digital age . fact-checking is an approach to assess the veracity of factual claims . a new dataset is presented to help advance automated fact- checking . |
| Approach: | They propose a dataset that assesses the veracity of factual claims using evidence from credible sources. |
| Outcome: | The proposed dataset can be used for automated fact-checking tasks. |
Step-by-Step Fact Verification System for Medical Claims with Explainable Reasoning (2025.naacl-short)
Copied to clipboard
| Challenge: | Fact verification (FV) aims to assess the veracity of a claim based on relevant evidence. |
| Approach: | They propose to use iterative fact verification to assess the veracity of a claim based on relevant evidence. |
| Outcome: | The proposed system improves on three medical fact-checking datasets and evaluates with multiple settings including different LLMs, external web search, and structured reasoning using logic predicates. |
Scientific Fact-Checking: A Survey of Resources and Approaches (2023.findings-acl)
Copied to clipboard
| Challenge: | Fact-checking is the task of assessing the veracity of factual claims based on credible evidence and background knowledge. |
| Approach: | They propose to automate scientific fact-checking using natural language processing to assess the veracity of factual claims based on credible evidence and background knowledge. |
| Outcome: | The proposed methods can help combat the spread of misinformation and help individuals understand new scientific breakthroughs. |
DP-MLM: Differentially Private Text Rewriting Using Masked Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for text privatization using Differential Privacy rely on autoregressive models which lack a mechanism to contextualize the private rewriting process. |
| Approach: | They propose a method for differentially private text rewriting using masked language models to rewrite a text one token at a time. |
| Outcome: | The proposed method preserves utility at lower levels, compared to previous methods relying on autoregressive models with a decoder. |