Papers by William Hinthorn
PROMPTEVALS: A Dataset of Assertions and Guardrails for Custom Production Large Language Model Pipelines (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models fail to follow instructions or meet developer expectations when running in production . a dataset of 2087 LLM pipeline prompts with 12623 assertion criteria is larger than previous collections . |
| Approach: | They propose a dataset of 2087 LLM pipeline prompts with 12623 assertion criteria . they fine-tuned Mistral and Llama 3 models outperform GPT-4o by 20.93% on average . |
| Outcome: | The proposed dataset outperforms GPT-4o and mistral models in generating assertions and offers reduced latency and improved performance. |
Enhancing Factual Consistency of Abstractive Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Abstractive summarization models often distort or fabricate facts in articles . factual inconsistency is a common problem with abstractive summaries . |
| Approach: | They propose a fact-aware summarization model FASum to extract factual relations into the summary generation process via graph attention. |
| Outcome: | The proposed model can produce abstractive summaries with higher factual consistency compared with existing systems and corrects factual errors via modifying only a few keywords. |