| Challenge: | Document and discourse segmentation are two fundamental NLP tasks pertaining to breaking up text into constituents. |
| Approach: | They propose three transformer-based NLP models that break up text into constituents and compare them to previous approaches. |
| Outcome: | The proposed architectures reduce errors by a large margin on three datasets and improve performance on real-world datasets. |
Similar Papers
Neural Document Segmentation Using Weighted Sliding Windows with Transformer Encoders (2025.coling-industry)
Copied to clipboard
| Challenge: | Using overlapping text sequences and position-aware weighting, we achieve up to a 10% increase in segmentation F1 score compared to existing methods. |
| Approach: | They propose a Transformer-based method for document segmentation that utilizes overlapping text sequences with a unique position-aware weighting mechanism to enhance segmentation accuracy. |
| Outcome: | The proposed method achieves up to 10% increase in segmentation F1 score compared to existing methods and improves quality of generated responses by 5% while achieving four times greater efficiency. |
Exploiting Discourse-Level Segmentation for Extractive Summarization (D19-54)
Copied to clipboard
| Challenge: | Existing approaches to extract summarize text are based on sentences as the elementary unit, but semantic segments containing supplementary information or descriptive details are often nonessential in the generated summaries. |
| Approach: | They propose to exploit discourse-level segmentation as a finer-grained means to more precisely pinpoint the core content in a document. |
| Outcome: | The proposed method improves extractive summarization performance on CNN/Daily Mail dataset. |
Transformer over Pre-trained Transformer for Neural Text Segmentation with Enhanced Topic Coherence (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for text segmentation use supervised and unsupervised learning to perform tasks such as text summarization and keyword extraction. |
| Approach: | They propose a transformer over transformer framework to perform neural text segmentation. |
| Outcome: | The proposed framework outperforms state-of-the-art models in terms of semantic coherence measure . bottom-level sentence encoders pre-trained on specific languages yield better performance . |
Hierarchical Transformers for Multi-Document Summarization (P19-1)
Copied to clipboard
| Challenge: | Existing models for multidocument summarization have been developed that can process multiple documents in a hierarchical manner. |
| Approach: | They propose a neural summarization model which can process multiple input documents and distill Transformer architecture with the ability to encode documents in a hierarchical manner. |
| Outcome: | The proposed model improves on the WikiSum dataset and can process multiple documents in a hierarchical manner. |
Revisiting Transformer-based Models for Long Document Classification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent literature in text classification is biased towards short text sequences . multi-page multi-paragraph documents cannot be efficiently encoded by vanilla transformers based on short text. |
| Approach: | They compare different Transformer-based Long Document Classification approaches to mitigate the computational overhead of vanilla transformers to encode much longer text. |
| Outcome: | The proposed models can process longer text and provide practical advice for long document classification tasks. |
Toward Unifying Text Segmentation and Long Document Summarization (2022.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive strategies produce more condensed summaries, but they suffer from hallucinations and factual errors, which pose a more difficult generation challenge. |
| Approach: | They propose a method that learns robust sentence representations by performing summarization and segmentation simultaneously, which is further enhanced by an optimization-based regularizer to promote selection of diverse summary sentences. |
| Outcome: | The proposed model achieves state-of-the-art performance on publicly available benchmarks and better cross-genre transferability when equipped with text segmentation. |
Discourse Representation Parsing for Sentences and Documents (P19-1)
Copied to clipboard
| Challenge: | Experimental results show that our model outperforms competitive baselines by a wide margin. |
| Approach: | They propose a neural model which parses discourse structures of arbitrary length and granularity. |
| Outcome: | The proposed model outperforms baseline models on sentence- and document-level benchmarks. |
Text Segmentation as a Supervised Learning Task (N18-2)
Copied to clipboard
| Challenge: | Existing datasets for text segmentation are small in size and do not represent the natural distribution of text in documents. |
| Approach: | They propose a large dataset for text segmentation that is automatically extracted and labeled from Wikipedia and develop a model based on this dataset. |
| Outcome: | The proposed model generalizes well to unseen natural text. |
Top-down Discourse Parsing via Sequence Labelling (2021.eacl-main)
Copied to clipboard
| Challenge: | Discourse analysis is a systematic way to understand how texts are segmented hierarchically into discourse units. |
| Approach: | They propose a top-down approach to discourse parsing that is conceptually simpler than its predecessors. |
| Outcome: | The proposed model eliminates the decoder and reduces the search space for splitting points. |
Toward Fast and Accurate Neural Discourse Segmentation (D18-1)
Copied to clipboard
| Challenge: | Existing discourse segmenters rely on complicated hand-crafted features and are not practical in actual use. |
| Approach: | They propose an end-to-end neural segmenter based on BiLSTM-CRF framework that can segment texts fast and accurately using a large corpus. |
| Outcome: | The proposed model is significantly faster than previous methods while achieving state-of-the-art performance on the RST-DT corpus. |