| Challenge: | Discourse structures are a relational semantic structure that convey causal, topical, argumentative relations or more generally coherence relations. |
| Approach: | They propose to use Snorkel to label training data using expert-composed heuristics and transform them into probability distributions of the class labels given to training candidates. |
| Outcome: | The proposed paradigm can be used for difficult tasks such as that of discourse attachment. |
Similar Papers
Weak Supervision for Learning Discourse Structure (D19-1)
Copied to clipboard
| Challenge: | a weak supervision approach is a promising tool for learning discourse structure for multi-party dialogue. |
| Approach: | They propose a data programming paradigm that allows a user to label training data using expert-composed heuristics and transform them into probability distributions of the class labels. |
| Outcome: | The proposed approach outperforms both deep learning and traditional ML approaches on the task of learning discourse structure for multi-party dialogue. |
Learning with Limited Text Data (2022.acl-tutorials)
Copied to clipboard
| Challenge: | Natural Language Processing (NLP) relies on labeled data to perform state-of-the-art performance . labeles are often required to label large amounts of textual data . this tutorial will provide an overview of labeleing in NLP . |
| Approach: | This tutorial will provide a systematic overview of methods for learning from limited labeled data. |
| Outcome: | This tutorial will provide a systematic and up-to-date overview of the proposed methods . it will highlight current challenges and future directions . |
Evaluating Discourse in Structured Text Representations (P19-1)
Copied to clipboard
| Challenge: | Discourse structure is integral to understanding a text and is useful in many NLP tasks. |
| Approach: | They propose a structured attention mechanism for text classification that derives a tree over a text, akin to an RST discourse tree. |
| Outcome: | The proposed model improves performance on multiple discourse-relevant tasks and datasets and ablation studies show it does little to capture discourse structure. |
The Strength of the Weakest Supervision: Topic Classification Using Class Labels (N19-3)
Copied to clipboard
| Challenge: | a topic classifier can understand only class labels when training for tasks that require a large amount of labeled documents. |
| Approach: | They propose an algorithm that can initialize a topic classifier using only class labels . they propose a method that combines word embedding and naive Bayes classification . |
| Outcome: | The proposed approach saves significant initial labeling effort by providing a "warm start" the proposed approach can be fine-tuned with more labeled documents to reach a certain performance level. |
Putting Context in Context: the Impact of Discussion Structure on Text Classification (2024.eacl-long)
Copied to clipboard
| Challenge: | Current text classification approaches focus on the content to be classified, but contextual information is neglected in many cases. |
| Approach: | They propose to integrate contextual information into a transformer-based model by feeding it as natural language input into . they also experiment with different amounts of training data and analyse local discussion networks in a privacy-compliant way. |
| Outcome: | The proposed model can be generalized to other datasets and is privacy-compliant. |
Augmenting BERT-style Models with Predictive Coding to Improve Discourse-level Representations (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing language models do not produce suitable representations at the discourse level. |
| Approach: | They propose to augment BERT-style language models with a mechanism that allows them to learn suitable discourse-level representations by incorporating top-down connections that operate at the intermediate layers of the network. |
| Outcome: | The proposed approach improves in 6 out of 11 tasks by detecting discourse relationship detection. |
Discourse Representation Parsing for Sentences and Documents (P19-1)
Copied to clipboard
| Challenge: | Experimental results show that our model outperforms competitive baselines by a wide margin. |
| Approach: | They propose a neural model which parses discourse structures of arbitrary length and granularity. |
| Outcome: | The proposed model outperforms baseline models on sentence- and document-level benchmarks. |
Data, Data Everywhere: A Guide for Pretraining Dataset Construction (2024.emnlp-main)
Copied to clipboard
Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings, Bo Liu, Aastha Jhunjhunwala, Zhilin Wang, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
| Challenge: | Recent language models have impressive capabilities on a number of evaluation areas. |
| Approach: | They conduct systematic analysis of pretraining set construction to identify which methods yield the greatest gains in model accuracy. |
| Outcome: | The proposed method can be used to refine and improve a pretraining set. |
Rethinking Data Mixture for Large Language Models: A Comprehensive Survey and New Perspectives (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for data mixture improve the generalization capability of large language models (LLMs) on downstream tasks. |
| Approach: | They propose a fine-grained categorization of existing methods and propose three subtypes of offline and online methods. |
| Outcome: | The proposed methods extend beyond offline and online classifications and highlight key challenges in the field of data mixture. |
Which *BERT? A Survey Organizing Contextualized Encoders (2020.emnlp-main)
Copied to clipboard
| Challenge: | a survey on language representation learning aims to highlight common themes . we focus on the areas of progress, compared to other fields, and discuss how each area is evaluated. |
| Approach: | They present a survey on language representation learning to highlight common themes . they compare contributions in contextualized text encoders to ideas from other fields . |
| Outcome: | The proposed survey aims to highlight common themes in the field of language representation learning. |