Papers by Bhaktipriya Radharapu
Calibrating LLM Judges: Linear Probes for Fast and Reliable Uncertainty Estimation (2026.acl-industry)
Copied to clipboard
Bhaktipriya Radharapu, Eshika Saxena, Kenneth Li, Chenxi Whitehouse, Adina Williams, Nicola Cancedda
| Challenge: | Existing methods for obtaining well-calibrated uncertainty estimates are poorly calibrated or computationally expensive. |
| Approach: | They propose a linear probe that provides calibrated uncertainty estimates from reasoning judges’ hidden states, requiring no additional model training. |
| Outcome: | The proposed method achieves superior calibration compared to existing methods with x computational savings, generalizes robustly to unseen evaluation domains, and delivers higher accuracy on high-confidence predictions. |
AART: AI-Assisted Red-Teaming with Diverse Data Generation for New LLM-powered Applications (2023.emnlp-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are rapidly becoming more and more popular, but dealing with the potential harms associated with their deployment in real-world scenarios is still an open research question. |
| Approach: | They propose an automated approach for automated generation of adversarial evaluation datasets to test the safety of LLM generations on new downstream applications. |
| Outcome: | AART generates evaluation datasets with high diversity of content characteristics critical for effective adversarial testing. |
Debiasing Text Safety Classifiers through a Fairness-Aware Ensemble (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Increasing use of large language models (LLMs) require performant guardrails to ensure the safety of inputs and outputs . when these guardrail are trained on imbalanced data, they can learn the societal biases resulting from the model's performance. |
| Approach: | They propose a method for mitigating counterfactual fairness in closed-source text safety classifiers by using a debiasing regularizer and a threshold-agnostic metric. |
| Outcome: | The proposed method outperforms classifiers and acts as a debiasing regularizer . it uses threshold-agnostic metrics and Fair Data Reweighting (FDW) to assess the counterfactual fairness of a model . |
Arbiters of Ambivalence: Challenges of using LLMs in No-Consensus tasks (2025.findings-acl)
Copied to clipboard
| Challenge: | LLMs are increasingly being used to replace humans in "aligning" LLM training . studies question this trend, but have found they can be more effective in ambivalent scenarios where humans disagree . |
| Approach: | They develop a “no-consensus” benchmark by curating examples that encompass a variety of a priori ambivalent scenarios. |
| Outcome: | The proposed benchmarks show that LLMs can provide nuanced assessments when generating open-ended answers, but tend to take a stance on no-consensus topics when employed as judges or debaters. |