Papers by Jonathan Bragg
ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews (2024.acl-long)
Copied to clipboard
| Challenge: | Existing systems that can interpret complex writing feedback and edit documents in response are limited on the most demanding writing tasks. |
| Approach: | They propose to use peer feedback to revise scientific papers based on peer feedback . they provide labels linking each reviewer comment to the specific paper edits made by the author . |
| Outcome: | The proposed model fails to identify which edits correspond to a comment and the original paper. |
Re-evaluating Automatic LLM System Ranking for Alignment with Human Preference (2025.findings-naacl)
Copied to clipboard
| Challenge: | Evaluating and ranking the capabilities of different LLMs is crucial for understanding their performance and alignment with human preferences. |
| Approach: | They propose a system-level evaluation framework that ranks LLMs based on their alignment with human preferences. |
| Outcome: | The proposed framework aims to rank LLMs based on their performance and alignment with human preferences. |
GENIE: Toward Reproducible and Standardized Human Evaluation for Text Generation (2022.emnlp-main)
Copied to clipboard
Daniel Khashabi, Gabriel Stanovsky, Jonathan Bragg, Nicholas Lourie, Jungo Kasai, Yejin Choi, Noah A. Smith, Daniel Weld
| Challenge: | Effective human evaluation of text generation tasks remains an important, open area for research. |
| Approach: | They propose a system for running standardized human evaluations across different generation tasks. |
| Outcome: | The proposed system produces standardized human evaluations across tasks . it crowdsources predictions and ranks systems on leaderboards . the proposed system is not reproducible over time and different annotator populations . |