Papers by Benjamin Newman
PaperMage: A Unified Toolkit for Processing, Representing, and Manipulating Visually-Rich Scientific Documents (2023.emnlp-demo)
Copied to clipboard
Kyle Lo, Zejiang Shen, Benjamin Newman, Joseph Chang, Russell Authur, Erin Bransom, Stefan Candra, Yoganand Chandrasekhar, Regan Huff, Bailey Kuehl, Amanpreet Singh, Chris Wilhelm, Angele Zamarron, Marti A. Hearst, Daniel Weld, Doug Downey, Luca Soldaini
| Challenge: | Existing tools for working with scientific documents are limited and documents are often in difficult-to-use PDF formats. |
| Approach: | They propose an open-source Python toolkit for analyzing and processing visually-rich scientific documents. |
| Outcome: | PaperMage provides turn-key recipes for common scientific document processing use-cases. |
Refining Targeted Syntactic Evaluation of Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Targeted syntactic evaluation of subject-verb number agreement in English (TSE) evaluates language models’ syntaktic knowledge using hand-crafted minimal pairs of sentences that differ only in the main verb’s conjugation. |
| Approach: | They propose to use templates to evaluate language models' syntactic knowledge to assess their ability to conjugate arbitrary verbs and their likely behavior to measure their likelihood of conjugating grammatical sentences. |
| Outcome: | The proposed evaluations overestimate systematicity of language models, but score up to 40% better on verbs that they predict are likely in context. |
ArxivDIGESTables: Synthesizing Scientific Literature into Tables using Language Models (2024.emnlp-main)
Copied to clipboard
Benjamin Newman, Yoonjoo Lee, Aakanksha Naik, Pao Siangliulue, Raymond Fok, Juho Kim, Daniel Weld, Joseph Chee Chang, Kyle Lo
| Challenge: | Using language models (LMs) can generate literature review tables by decomposing it into separate schema and value generation steps. |
| Approach: | They propose a framework that leverages language models to perform literature review table generation by decomposing it into separate schema and value generation steps. |
| Outcome: | The proposed framework decomposes the task into two sub-tasks: schema generation and value generation. |
A Question Answering Framework for Decontextualizing User-facing Snippets from Scientific Documents (2023.emnlp-main)
Copied to clipboard
| Challenge: | snippets are not meant to be read outside their original document. |
| Approach: | They propose a framework that decomposes the task into three stages: question generation, question answering, and rewriting. |
| Outcome: | The proposed framework decomposes the task into three stages: question generation, question answering, and rewriting. |