Papers by Anastasiia Sedova
To Know or Not To Know? Analyzing Self-Consistency of Large Language Models under Ambiguity (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have remarkable performance in a variety of tasks due to factual knowledge accumulated during pre-training. |
| Approach: | They propose an evaluation protocol that disentangles knowing from applying knowledge and test state-of-the-art LLMs on 49 ambiguous entities. |
| Outcome: | The proposed evaluation protocol disentangles knowing from applying knowledge and tests state-of-the-art LLMs on 49 ambiguous entities. |
ULF: Unsupervised Labeling Function Correction using Cross-Validation for Weak Supervision (2023.emnlp-main)
Copied to clipboard
| Challenge: | A cost-effective alternative to manual data labeling is weak supervision (WS), where data samples are automatically annotated using a predefined set of labeling functions (LFs). |
| Approach: | They propose an algorithm which denoises WS data by leveraging models trained on all but some LFs to identify and correct biases specific to the held-out LF. |
| Outcome: | The proposed algorithm denoises WS data by leveraging models trained on all but some LFs to identify and correct biases specific to the held-out LF. |
ACTC: Active Threshold Calibration for Cold-Start Knowledge Graph Completion (2023.acl-short)
Copied to clipboard
| Challenge: | Knowledge graphs are a graph of information organized as entities, relations, and entities. |
| Approach: | They propose a method to calibrate a scoring model over (entity, relation, entity)-tuples . they use an annotated set of tuple truncated by Logistic Regression or Gaussian Process classifiers . |
| Outcome: | The proposed method finds good per-relation thresholds efficiently based on a limited set of annotated tuples. |