Papers by Samuel Ackerman
Reliable and Interpretable Drift Detection in Streams of Short Texts (2023.acl-industry)
Copied to clipboard
| Challenge: | Data drift is a key factor leading to model performance degradation over time. |
| Approach: | They propose a framework for reliable model-agnostic change-point detection and interpretation in large task-oriented dialog systems. |
| Outcome: | The proposed framework is effective in multiple customer deployments. |
A Novel Metric for Measuring the Robustness of Large Language Models in Non-adversarial Scenarios (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Using large language models, we evaluated their robustness on multiple datasets. |
| Approach: | They propose a new metric for assessing model robustness by empirical evaluation of several models on multiple datasets. |
| Outcome: | The proposed metric is based on a set of datasets that are constructed by introducing naturally-occurring, non-malicious perturbations or by generating semantically equivalent paraphrases of input questions or statements. |