DisSent: Learning Sentence Representations from Explicit Discourse Relations (P19-1)
Copied to clipboard
| Challenge: | Existing models train on vast amounts of text or require costly, manually curated datasets. |
| Approach: | They propose to leverage the discourse relations between sentences to curate a high quality sentence relation task by leveraging explicit discourse relations. |
| Outcome: | The proposed model can be used to learn the meaning of two sentences in a bidirectional LSTM sentence encoder. |
Similar Papers
Next Sentence Prediction helps Implicit Discourse Relation Classification within and across Domains (D19-1)
Copied to clipboard
| Challenge: | Discourse relation classification is one of the most difficult tasks in discourse parsing. |
| Approach: | They propose a bidirectional encoder representation from transformer model that encodes a representation of likely next sentences. |
| Outcome: | The proposed model outperforms the state-of-the-art system in 11-way classification by 8% points on the standard PDTB dataset. |
Deep Enhanced Representation for Implicit Discourse Relation Recognition (C18-1)
Copied to clipboard
| Challenge: | Discourse parsing requires understanding of text spans and can't be easily derived from surface features from sentence pairs. |
| Approach: | They propose a model augmented with different grained text representations to improve discourse relation recognition. |
| Outcome: | The proposed model achieves state-of-the-art accuracy with greater than 48% in 11-way and F1 score greater than 50% in 4-way classifications for the first time according to our best knowledge. |
Employing the Correspondence of Relations and Connectives to Identify Implicit Discourse Relations via Label Embeddings (P19-1)
Copied to clipboard
| Challenge: | Existing models for implicit discourse relation recognition lack the ability to accurately map connectives into discourse relations. |
| Approach: | They propose a multi-task learning framework where relations and connectives are simultaneously predicted and leveraged to transfer knowledge between the two prediction tasks. |
| Outcome: | The proposed framework yields state-of-the-art performance on several settings of the Penn Discourse Treebank dataset. |
SLM: Learning a Discourse Language Representation with Sentence Unshuffling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent models for learning discourse language representations focus on bottom or top-level representations, but they do not capture intermediate-size structures in natural languages such as sentences and the relationships among them. |
| Approach: | They propose a new objective for learning a discourse language representation in a self-supervised manner by shuffling the sequence of input sentences and training a hierarchical transformer model to reconstruct the original ordering. |
| Outcome: | The proposed model improves the original BERT model on downstream tasks by large margins. |
Adapting BERT to Implicit Discourse Relation Classification with a Focus on Discourse Connectives (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing studies on the performance of BERT for implicit discourse relation classification have not been conducted. |
| Approach: | They propose to apply BERT to implicit discourse relation classification by performing additional pre-training on text tailored to discourse relations. |
| Outcome: | The proposed methods outperform previous state-of-the-art models in many tasks. |
TransS-Driven Joint Learning Architecture for Implicit Discourse Relation Recognition (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to implicit discourse relation recognition lack connectives as strong linguistic clues. |
| Approach: | They propose a transS-driven joint learning architecture to translate discourse relations in low-dimensional embedding space and exploit the semantic features of arguments to assist discourse understanding. |
| Outcome: | The proposed model outperforms existing systems on the Penn Discourse TreeBank. |
A Comprehensive Survey of Sentence Representations: From the BERT Epoch to the CHATGPT Era and Beyond (2024.eacl-long)
Copied to clipboard
Abhinav Ramesh Kashyap, Thanh-Tung Nguyen, Viktor Schlegel, Stefan Winkler, See-Kiong Ng, Soujanya Poria
| Challenge: | Sentence representations are a critical component in NLP applications such as retrieval, question answering, and text classification. |
| Approach: | They present a systematic review of the literature on sentence representations focusing mostly on deep learning models. |
| Outcome: | The proposed methods highlight the key contributions and challenges in this area and suggest potential avenues for improving the quality and efficiency of sentence representations. |
Discourse Representation Parsing for Sentences and Documents (P19-1)
Copied to clipboard
| Challenge: | Experimental results show that our model outperforms competitive baselines by a wide margin. |
| Approach: | They propose a neural model which parses discourse structures of arbitrary length and granularity. |
| Outcome: | The proposed model outperforms baseline models on sentence- and document-level benchmarks. |
What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties (P18-1)
Copied to clipboard
| Challenge: | a lack of understanding of the properties of sentence embeddings is limiting the use of the techniques. |
| Approach: | They propose 10 probing tasks designed to capture simple linguistic features of sentences . they use three different encoders to train embeddings in eight different ways . |
| Outcome: | The proposed tasks capture key linguistic features of sentences, but they are difficult to infer from them. |
Enhancing Reasoning Capabilities by Instruction Learning and Chain-of-Thoughts for Implicit Discourse Relation Recognition (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for implicit discourse relation recognition are based on generative models, but some studies suggest they do not perform as well as generic encoder-only models for NLU tasks. |
| Approach: | They propose a classification method that is solely based on generative models and utilize Chain-of-Thoughts to partition the inference process into a sequence of three successive stages. |
| Outcome: | The proposed model outperforms existing models on a natural language understanding task. |