| Challenge: | a new syntactic representation that commits to syntakic choices is proposed for humans . we use a system that uses only incremental processing of a prefix to predict the word in a sentence . |
| Approach: | They propose a syntactic representation that commits to syntakic choices incrementally . they say the system can achieve 93.72 F1 on the Penn Treebank with as few as 5 bits per word . |
| Outcome: | The proposed representation achieves 93.72 F1 on the Penn Treebank with as few as 5 bits per word . the analysis of the representations shows they have interpretable features and deferred resolution of syntactic ambiguities. |
Similar Papers
From Partial to Strictly Incremental Constituent Parsing (2024.eacl-short)
Copied to clipboard
| Challenge: | Incremental NLP aims to learn and adapt partial representations as information unfolds, but studies on incremental approaches have focused on non-incremental approaches. |
| Approach: | They build incremental constituent parsers that adhere to a strong definition of incrementality across languages and compare them against non-incremental and partially incremental models. |
| Outcome: | The proposed models are compared against non-incremental and partially incremental models. |
Overestimation of Syntactic Representation in Neural Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Several testing methodologies have been developed to probe models’ syntactic representations. |
| Approach: | They propose a method to determine syntactic structure by training a model on strings generated according to a template and testing its ability to distinguish between similar ones with different syntax. |
| Outcome: | The proposed method reproduces positive results with two non-syntactic baseline language models: an n-gram model and an LSTM model trained on scrambled inputs. |
Advances in Pre-Training Distributed Word Representations (L18-1)
Copied to clipboard
| Challenge: | Pre-trained word representations are a building block of many Natural Language Processing and Machine Learning applications. |
| Approach: | They propose to combine known tricks and a set of publicly available pre-trained word vector representations to train high-quality representations. |
| Outcome: | The proposed models outperform the current state of the art on a number of tasks while maintaining a high training speed to scale to massive amount of data. |
Unifying Parsing and Tree-Structured Models for Generating Sentence Semantic Representations (2022.naacl-srw)
Copied to clipboard
| Challenge: | Existing tree-based models require handannotated data to be trained. |
| Approach: | They propose a tree-based model that learns its composition function together with its structure. |
| Outcome: | The proposed model outperforms existing models on downstream tasks and is competitive with Bert base model. |
AMR Parsing via Graph-Sequence Iterative Inference (2020.acl-main)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) parsing is a broad-coverage semantic formalism that encodes the meaning of a sentence as a rooted, directed, labeled graph. |
| Approach: | They propose a model that treats AMR parsing as a series of dual decisions on the input sequence and the incrementally constructed graph. |
| Outcome: | The proposed model outperforms existing models by large margins on both input sequence and output graph. |
Unsupervised Cross-Lingual Representation Learning (P19-4)
Copied to clipboard
| Challenge: | a comprehensive survey of cutting-edge weakly-supervised and unsupervised cross-lingual word representations is presented . |
| Approach: | This tutorial provides a comprehensive survey of recent work on weakly-supervised and unsupervised cross-lingual word representations. |
| Outcome: | This tutorial provides a comprehensive survey of cutting-edge weakly-supervised and unsupervised word representations. |
CCG Parsing Algorithm with Incremental Tree Rotation (N19-1)
Copied to clipboard
| Challenge: | Combinatory Categorial Grammar (CCG) is a mildly context sensitive grammar formalism that excels in incremental sentence processing. |
| Approach: | They propose a new incremental parsing algorithm that uses a syntactic approach . it uses right-branching constituent structures and optional constituents that adjoin on the right . |
| Outcome: | The proposed algorithm can cover the whole CCGbank with greater incrementality and accuracy than previous proposals. |
Learning Visually Grounded Sentence Representations (N18-1)
Copied to clipboard
| Challenge: | Unsupervised sentence representation models suffer from the grounding problem because of lack of association between symbols and external information. |
| Approach: | They train a sentence encoder to predict image features of a caption and use them as sentence representations. |
| Outcome: | The proposed model improves on word embeddings and word representations on standard benchmarks. |
Unsupervised Natural Language Parsing (Introductory Tutorial) (2021.eacl-tutorials)
Copied to clipboard
| Challenge: | Unsupervised parsing learns a syntactic parser from training sentences without parse tree annotations. |
| Approach: | This tutorial will introduce what unsupervised parsing does and how it can be useful for and beyond syntactic parse. |
| Outcome: | This paper will provide an overview of major approaches to unsupervised parsing and analyze their strengths and weaknesses. |
Which *BERT? A Survey Organizing Contextualized Encoders (2020.emnlp-main)
Copied to clipboard
| Challenge: | a survey on language representation learning aims to highlight common themes . we focus on the areas of progress, compared to other fields, and discuss how each area is evaluated. |
| Approach: | They present a survey on language representation learning to highlight common themes . they compare contributions in contextualized text encoders to ideas from other fields . |
| Outcome: | The proposed survey aims to highlight common themes in the field of language representation learning. |