Papers by Tara Safavi
Evaluating the Calibration of Knowledge Graph Embeddings for Trustworthy Link Prediction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing calibration techniques are less effective under the standard closed-world assumption (CWA) and the more realistic open-world hypothesis (OWA) Existing methods are not effective under OWA and provide explanations for this discrepancy. |
| Approach: | They conduct an evaluation under the standard closed-world assumption (CWA) and introduce the more realistic but challenging open-world assume (OWA) . they find existing calibration techniques are much less effective under the OWA than the CWA . |
| Outcome: | The proposed calibration techniques are much less effective under the open-world assumption (OWA) and explain the discrepancy. |
NegatER: Unsupervised Discovery of Negatives in Commonsense Knowledge Bases (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in knowledge base construction techniques focus on the acquisition of positive (true) KB statements, but negative (false) statements are important for discriminative reasoning. |
| Approach: | They propose a framework that ranks potential negatives in commonsense KBs using a contextual language model. |
| Outcome: | The proposed framework ranks negatives in commonsense KBs using a language model . it yields positives that are more grammatical, coherent, and informative . |
Interpretable User Satisfaction Estimation for Conversational Systems with Large Language Models (2024.acl-long)
Copied to clipboard
Ying-Chun Lin, Jennifer Neville, Jack Stokes, Longqi Yang, Tara Safavi, Mengting Wan, Scott Counts, Siddharth Suri, Reid Andersen, Xiaofeng Xu, Deepak Gupta, Sujay Kumar Jauhar, Xia Song, Georg Buscher, Saurabh Tiwary, Brent Hecht, Jaime Teevan
| Challenge: | Existing approaches to user satisfaction estimation are hard to interpret and lack generalizable patterns. |
| Approach: | They propose to use supervised prompting to extract interpretable user satisfaction signals from natural language utterances to tailor an LLM to USE using labeled examples. |
| Outcome: | The proposed method extracts interpretable signals of user satisfaction from natural language utterances more effectively than embedding-based approaches. |
Relational World Knowledge Representation in Contextual Language Models: A Review (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge bases are organized according to manual schemas that limit their expressiveness and require significant human engineering and maintenance. |
| Approach: | They propose to organize knowledge representation strategies in LMs by the level of KB supervision provided . they propose to highlight notable models, evaluation tasks, and findings . |
| Outcome: | The proposed model can internalize and express relational knowledge in more flexible forms. |
CoDEx: A Comprehensive Knowledge Graph Completion Benchmark (2020.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge graph completion benchmarks for knowledge graphs are often incomplete . however, the field has remained static over the past decade . |
| Approach: | They propose to use Wikidata and Wikipedia to improve on existing benchmarks . they analyze logical relation patterns, then perform baseline link prediction and triple classification . |
| Outcome: | The proposed datasets improve upon existing benchmarks in scope and difficulty. |
S3-DST: Structured Open-Domain Dialogue Segmentation and State Tracking in the Era of LLMs (2024.findings-acl)
Copied to clipboard
Sarkar Snigdha Sarathi Das, Chirag Shah, Mengting Wan, Jennifer Neville, Longqi Yang, Reid Andersen, Georg Buscher, Tara Safavi
| Challenge: | Dialogue state tracking (DST) was based on narrow task-oriented conversations . however, large language models have ushered in more flexible open-domain chat systems . |
| Approach: | They propose a method that combines dialogue segmentation and state tracking within open-domain dialogues to improve long context tracking. |
| Outcome: | The proposed method outperforms the state-of-the-art on open-domain dialogue datasets and publicly available datasets. |