Papers by Jake Smith
What are the Desired Characteristics of Calibration Sets? Identifying Correlates on Long Form Scientific Summarization (2023.acl-long)
Copied to clipboard
Griffin Adams, Bichlien Nguyen, Jake Smith, Yingce Xia, Shufang Xie, Anna Ostropolets, Budhaditya Deb, Yuan-Jyue Chen, Tristan Naumann, Noémie Elhadad
| Challenge: | Summarization models are trained to maximize the likelihood of a single reference (MLE) but little is known about why one setup is more effective than another . |
| Approach: | They add a calibration step which exposes a model to its own ranked outputs to improve relevance or contrasts positive and negative sets to improve faithfulness. |
| Outcome: | The proposed calibration step can unlock large gains in relevance or faithfulness. |