Papers by Robert Morabito
Fine-Tuned LLMs are “Time Capsules” for Tracking Societal Bias Through Books (2025.naacl-long)
Copied to clipboard
| Challenge: | We develop a corpus comprising 593 fictional books across seven decades (1950-2019) to track bias evolution. |
| Approach: | They develop a method to trace and quantify bias evolution using fine-tuned LLMs on fictional books across seven decades to track bias evolution. |
| Outcome: | The proposed method traces and quantifies bias evolution in a corpus of 593 fictional books across seven decades. |
STOP! Benchmarking Large Language Models with Sensitivity Testing on Offensive Progressions (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that assess explicit and implicit biases are based on a single scenario . a dataset of 450 offensive progressions contains 2,700 sentences of varying severity . |
| Approach: | They evaluate a dataset of offensive progressions that contain 2,700 sentences . they find that even the best-performing models detect bias inconsistently . |
| Outcome: | The proposed dataset shows that even the best-performing models detect bias inconsistently . aligning models with human judgments on STOP can improve answer rates on sensitive tasks by 191% . |
Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used in high-stakes areas such as healthcare, law, and education. |
| Approach: | They propose a concept of Confidence-Probability Alignment that connects an LLM’s internal confidence to the confidence conveyed in the model’s response when explicitly asked about its certainty. |
| Outcome: | The proposed model shows the strongest confidence-probability alignment across a wide range of tasks. |
Debiasing should be Good and Bad: Measuring the Consistency of Debiasing Techniques in Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in deep learning have led to the creation of large Transformer-based language models (LMs). |
| Approach: | They propose a protocol which distinguishes methods that yield desirable results . they apply this protocol to a popular debiasing method, Self-Debiase, and compare it to one called Instructive Debiaser. |
| Outcome: | The proposed protocol provides essential insights into the generalizability and interpretability of debiasing methods that may otherwise go overlooked. |