Papers by Alexandra DeLucia
MedScore: Generalizable Factuality Evaluation of Open-ended Long-form Medical Answers by Domain-adapted Claim Decomposition and Verification (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing factuality evaluation pipelines are poor matches for medical domains . existing methods are limited to objective, entity-centric, formulaic texts . |
| Approach: | They propose a pipeline to decompose medical answers into condition-aware valid facts . they use a decomposition-then-verify approach to evaluate generated text . |
| Outcome: | The proposed method extracts up to three times as many valid facts as existing methods . the resulting factuality score substantially varies by decomposition method, corpus, and used backbone LLM . |
Bernice: A Multilingual Pre-trained Encoder for Twitter (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing language models for Twitter are monolingual, adapted from other domains, or trained on limited amount of in-domain data. |
| Approach: | They propose a multilingual RoBERTa language model that is trained from scratch on 2.5 billion tweets with a custom tweet-focused tokenizer. |
| Outcome: | The proposed model outperforms or matches models trained on monolingual and multilingual tweets on a variety of benchmarks and is more efficient compute- and data-wise to train completely on in-domain data with a specialized domain-specific tokenizer. |
Anti-LM Decoding for Zero-shot In-context Machine Translation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to zero-shot learning with large language models are poorly calibrated for zero-shoot tasks. |
| Approach: | They propose a contrastive decoding objective with a decay factor to address in-context bias . they conduct experiments on 3 model types and sizes, 3 language directions, and beam search . |
| Outcome: | The proposed method outperforms state-of-the-art decoding objectives with 20 BLEU points improvement from the default objective in some settings. |
Geo-Seq2seq: Twitter User Geolocation on Noisy Data through Sequence to Sequence Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | a new method for Twitter user geolocation rewrites noisy, multilingual location strings into structured English location names. |
| Approach: | They propose a sequence-to-sequence (seq2sequ) model that rewrites noisy location strings into structured English location names. |
| Outcome: | The proposed model can generalize well to unseen temporal data, but performance does vary by language and country. |
Common Law Annotations: Investigating the Stability of Dialog System Output Annotations (2023.findings-acl)
Copied to clipboard
Seunggun Lee, Alexandra DeLucia, Nikita Nangia, Praneeth Ganedi, Ryan Guan, Rubing Li, Britney Ngaw, Aditya Singhal, Shalaka Vaidya, Zijun Yuan, Lining Zhang, João Sedoc
| Challenge: | High agreement is often used to show reliability of annotation procedures, but it is insufficient to ensure or reproducibility. |
| Approach: | They propose a protocol that increases Inter-Annotator Agreement among annotators and a standardized and codified protocol that strictly enforces transparency in the annotation process. |
| Outcome: | The proposed protocol ensures transparency in the annotation process, which ensures reproducibility of annotation guidelines. |