Papers by Erin Bransom
CHIME: LLM-Assisted Hierarchical Organization of Scientific Studies for Literature Review Support (2024.findings-acl)
Copied to clipboard
Chao-Chun Hsu, Erin Bransom, Jenna Sparks, Bailey Kuehl, Chenhao Tan, David Wadden, Lucy Wang, Aakanksha Naik
| Challenge: | Literature review requires researchers to synthesize a large amount of information. |
| Approach: | They propose to use LLMs to generate hierarchical organizations from a set of studies . they use a human-in-the-loop process to correct errors in LLM-generated hierarchies . |
| Outcome: | The proposed model improves assignment of studies to categories by 12.6 F1 points. |
PaperMage: A Unified Toolkit for Processing, Representing, and Manipulating Visually-Rich Scientific Documents (2023.emnlp-demo)
Copied to clipboard
Kyle Lo, Zejiang Shen, Benjamin Newman, Joseph Chang, Russell Authur, Erin Bransom, Stefan Candra, Yoganand Chandrasekhar, Regan Huff, Bailey Kuehl, Amanpreet Singh, Chris Wilhelm, Angele Zamarron, Marti A. Hearst, Daniel Weld, Doug Downey, Luca Soldaini
| Challenge: | Existing tools for working with scientific documents are limited and documents are often in difficult-to-use PDF formats. |
| Approach: | They propose an open-source Python toolkit for analyzing and processing visually-rich scientific documents. |
| Outcome: | PaperMage provides turn-key recipes for common scientific document processing use-cases. |
LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization (2023.eacl-main)
Copied to clipboard
| Challenge: | Human evaluation is labor-intensive, expensive to scale, and difficult to design. |
| Approach: | They propose a set of guidelines for human evaluation of faithfulness in long-form summaries that address the following challenges: (1) How can we achieve high inter-annotator agreement on faithfulness scores? (2) How can our annotator minimize workload while maintaining accurate faithfulness? |
| Outcome: | The proposed framework reduces inter-annotator variance in faithfulness scores while minimizing annotator workload while maintaining accuracy. |
ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews (2024.acl-long)
Copied to clipboard
| Challenge: | Existing systems that can interpret complex writing feedback and edit documents in response are limited on the most demanding writing tasks. |
| Approach: | They propose to use peer feedback to revise scientific papers based on peer feedback . they provide labels linking each reviewer comment to the specific paper edits made by the author . |
| Outcome: | The proposed model fails to identify which edits correspond to a comment and the original paper. |
CARE: Extracting Experimental Findings From Clinical Literature (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing annotation schemas and datasets fail to capture real-world complexity and nuance of experimental findings. |
| Approach: | They propose a new annotation schema capturing fine-grained findings as n-ary relations between entities and attributes. |
| Outcome: | The proposed schema captures fine-grained findings as n-ary relations between entities and attributes. |
Personalized Jargon Identification for Enhanced Interdisciplinary Communication (2024.naacl-long)
Copied to clipboard
| Challenge: | Identifying and translating scientific jargon for individual researchers could speed up research, but current methods of jaron identification rely on corpus-level familiarity indicators rather than modeling researcher-specific needs. |
| Approach: | They collect over 10K term familiarity annotations from 11 computer science researchers and investigate supervised and prompt-based methods to predict individual jargon familiarity. |
| Outcome: | The proposed method improves jargon familiarity prediction by using domain, subdomain, and individual knowledge. |
S2abEL: A Dataset for Entity Linking from Scientific Tables (2023.emnlp-main)
Copied to clipboard
| Challenge: | Entity linking (EL) is a longstanding problem in natural language processing and information extraction. |
| Approach: | They propose a neural baseline method for EL on scientific tables containing many out-of-knowledge-base mentions and a method that significantly outperforms a generic table EL method. |
| Outcome: | The proposed method significantly outperforms state-of-the-art generic table EL method on scientific tables with many out-of knowledge-base mentions. |
SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositories (2024.emnlp-main)
Copied to clipboard
Ben Bogin, Kejuan Yang, Shashank Gupta, Kyle Richardson, Erin Bransom, Peter Clark, Ashish Sabharwal, Tushar Khot
| Challenge: | Large Language Models (LLMs) have made significant progress in writing code, but can they be used to reproduce results from research repositories? |
| Approach: | They propose a benchmark to evaluate the capability of Large Language Models to reproduce results from research repositories. |
| Outcome: | The benchmark aims to capture the realistic challenges faced by researchers working with machine learning and natural language processing repositories. |
Automated Metrics for Medical Multi-Document Summarization Disagree with Human Evaluations (2023.acl-long)
Copied to clipboard
Lucy Lu Wang, Yulia Otmakhova, Jay DeYoung, Thinh Hung Truong, Bailey Kuehl, Erin Bransom, Byron Wallace
| Challenge: | Prior work has shown that models may exploit shortcuts that are difficult to detect using standard n-gram similarity metrics such as ROUGE. |
| Approach: | They propose to use human-assessed summary quality facets and pairwise preferences to improve MDS evaluation methods. |
| Outcome: | The proposed methods improve the quality of literature review summarization models . they use human-assessed summary quality facets and pairwise preferences . |