| Challenge: | Recent work on pretrained language models has led to significant improvements in a range of NLP tasks. |
| Approach: | They propose a coherence model which interprets sentences incrementally to capture lexical relations between them. |
| Outcome: | The proposed model interprets sentences incrementally to capture lexical relations between them. |
Similar Papers
How coherent are neural models of coherence? (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to model coherence are limited to small newswire corpora . evaluators need to be trained on lexical and document levels to perform evaluations . |
| Approach: | They propose four generic evaluation tasks that capture coherence-specific properties . they aim at capturing correct use of discourse connectives and lexical cohesion . |
| Outcome: | The proposed tasks capture coherence-specific properties, including correct use of discourse connectives, lexical cohesion, temporal consistency among events and participants in a story. |
Entity-based Neural Local Coherence Modeling (2022.acl-long)
Copied to clipboard
| Challenge: | Recent neural coherence models encode the input document using large-scale pretrained language models. |
| Approach: | They propose an entity-based neural local coherence model which is linguistically more sound than previous models. |
| Outcome: | The proposed model outperforms existing models on three downstream tasks. |
A Unified Neural Coherence Model (D19-1)
Copied to clipboard
| Challenge: | Existing models for coherence modeling fail on harder tasks with more realistic application scenarios. |
| Approach: | They propose a unified coherence model that incorporates sentence grammar, inter-sentence coherent relations, and global coherency patterns into a common neural framework. |
| Outcome: | The proposed model outperforms existing models on local and global discrimination tasks and outperformed existing models by a good margin. |
Coherence boosting: When your pretrained language model is not paying enough attention (2022.acl-long)
Copied to clipboard
| Challenge: | Long-range semantic coherence remains a challenge in automatic language generation and understanding. |
| Approach: | They propose a procedure that increases a model’s focus on a long context by distributional analyses of generated ordinary text and dialog responses. |
| Outcome: | The proposed procedure increases the model's focus on a long context. |
Pre-trained language model representations for language generation (N19-1)
Copied to clipboard
| Challenge: | Pre-trained language model representations have been successful in a wide range of language understanding tasks. |
| Approach: | They propose to use pre-trained language model representations to integrate them into sequence to sequence models and apply it to machine translation and abstractive summarization. |
| Outcome: | The proposed model is able to perform 5.3 BLEU in machine translation and 5.3 on the full text version of CNN/DailyMail. |
Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherence (2021.acl-short)
Copied to clipboard
| Challenge: | Recent neural topic models extract words from documents, but they are not coherent . coherence is crucial for topic models, but many use bag-of-words document representations as input . pre-trained language models are becoming ubiquitous in natural language processing . |
| Approach: | They combine contextualized representations with neural topic models to produce more coherent topics . they say that future improvements in language models will translate into better topic models . |
| Outcome: | The proposed approach produces more meaningful and coherent topics than bag-of-words models and recent neural models. |
Is Incoherence Surprising? Targeted Evaluation of Coherence Prediction from Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | a common approach to coherence evaluation is shuffling the sentence order of a text, creating incoherent text samples that need to be discriminated from the original. |
| Approach: | They propose an extendable set of test suites addressing different aspects of discourse and dialogue coherence. |
| Outcome: | The proposed evaluation paradigm is suited to evaluate linguistic qualities that contribute to the notion of coherence. |
Coherent or Not? Stressing a Neural Language Model for Discourse Coherence in Multiple Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work on coherence assessment using NLMs focuses on properties acquired from stand-alone sentences, but their ability to model discourse and pragmatic phenomena is still unclear. |
| Approach: | They propose to use a Neural Language Model to assess coherence in multiple languages to compare models' performance and to examine their performance in a cross-language scenario. |
| Outcome: | The proposed model can model coherent and incoherent text in multiple languages and in-domain settings. |
A Novel Computational Modeling Foundation for Automatic Coherence Assessment (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing models for text coherence assessment rely on a proxy task . however, this approach does not capture the full range of factors contributing to coherency. |
| Approach: | They propose a formal linguistic definition of what makes a discourse coherent and formalize these conditions as respective computational tasks that are jointly trained. |
| Outcome: | The proposed model improves on two human-rated coherence benchmarks. |
Incremental Processing in the Age of Non-Incremental Encoders: An Empirical Assessment of Bidirectional Models for Incremental NLU (2020.emnlp-main)
Copied to clipboard
| Challenge: | a number of languages are processed incrementally, but the best ones do not . we test five models on various datasets and compare their performance using three incremental evaluation metrics. |
| Approach: | They investigate how bidirectional LSTMs and Transformers behave under incremental interfaces . they propose to use bidirectional encoders in incremental mode while retaining non-incremental quality . |
| Outcome: | The proposed models perform better under incremental interfaces than the "omni-directional" BERT model, which achieves better non-incremental performance, but is impacted more by the incremental access. |