Papers by Mark Neumann
Knowledge Enhanced Contextual Word Representations (D19-1)
Copied to clipboard
Matthew E. Peters, Mark Neumann, Robert Logan, Roy Schwartz, Vidur Joshi, Sameer Singh, Noah A. Smith
| Challenge: | Existing methods to embed knowledge bases into large pre-training models do not contain any explicit grounding to real world entities and are difficult to recover factual knowledge. |
| Approach: | They propose a method to embed multiple knowledge bases (KBs) into large pretrained models with a Knowledge Attention and Recontextualization mechanism. |
| Outcome: | The proposed model improves perplexity, ability to recall facts and word sense disambiguation. |
S2ORC: The Semantic Scholar Open Research Corpus (2020.acl-main)
Copied to clipboard
| Challenge: | Academic papers are an increasingly important textual domain for natural language processing (NLP) research. |
| Approach: | They propose to aggregate 81.1M English-language academic papers into a unified source . they hope this resource will facilitate research and development of tools for text mining over academic text. |
| Outcome: | The proposed corpus includes metadata, abstracts, bibliographic references, and structured full text for 8.1M open access papers. |
Deep Contextualized Word Representations (N18-1)
Copied to clipboard
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, Luke Zettlemoyer
| Challenge: | a new type of deep contextualized word representation is proposed for language understanding problems . word vectors are learned functions of the internal states of a deep bidirectional language model . |
| Approach: | They propose a new type of deep contextualized word representation that models complex features of word use and how they vary across linguistic contexts. |
| Outcome: | The proposed representations improve the state of the art across six challenging NLP problems. |
Dissecting Contextual Word Embeddings: Architecture and Representation (D18-1)
Copied to clipboard
| Challenge: | Existing work on learning contextual representations has used LSTM-based biLMs, but there is no reason to believe this is effective. |
| Approach: | They propose to use pre-trained bidirectional language models to learn contextual word embeddings for four NLP tasks and to use them to study the effects of architecture on endtask accuracy. |
| Outcome: | The proposed models outperform word embeddings for four NLP tasks and all learn representations that vary with network depth. |
PAWLS: PDF Annotation With Labels and Structure (2021.acl-demo)
Copied to clipboard
| Challenge: | Existing tools for annotation of PDFs are limited to a web browser, allowing users to extract semantically meaningful regions from PDFs. |
| Approach: | They propose an annotation tool specifically designed for Adobe’s Portable Document Format (PDF) PAWLS supports span-based textual annotation, N-ary relations and freeform, non-textual bounding boxes. |
| Outcome: | The proposed tool supports span-based textual annotation, N-ary relations and freeform, non-textual bounding boxes. |
Writing Code for NLP Research (D18-3)
Copied to clipboard
| Challenge: | upcoming workshop on open source software for NLP aims to share best practices for writing code for Nl research . participants will learn how to write research code that facilitates good science and easy experimentation . |
| Approach: | this tutorial aims to share best practices for writing code for NLP research . participants will learn how to write research code that facilitates good science and easy debugging . |
| Outcome: | the workshop on open source software for NLP aims to share best practices for writing code for Nl research . participants will learn how to write research code that facilitates good science and easy experimentation . |