Papers with coherence
TRACE: An Experiential Framework for Coherent Multi-hop Knowledge Graph Question Answering (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for multihop Knowledge Graph Question Answering (KGQA) treat each reasoning step independently and fail to leverage experience from prior explorations, leading to fragmented reasoning and redundant exploration. |
| Approach: | They propose a framework that unifies LLM-driven contextual reasoning with exploration prior integration to enhance coherence and robustness of multihop KGQA. |
| Outcome: | Extensive experiments on multiple KGQA benchmarks show that TRACE outperforms state-of-the-art methods. |
Reference-free Summarization Evaluation via Semantic Correlation and Compression Ratio (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing evaluation metrics for summarization use human annotations as reference. |
| Approach: | They propose a new automatic reference-free evaluation metric that compares semantic distribution between source document and summary by pretrained language models and considers summary compression ratio. |
| Outcome: | The proposed metric is more consistent with human evaluation in terms of coherence, consistency, relevance and fluency. |
How to Evaluate a Summarizer: Study Design and Statistical Analysis for Manual Linguistic Quality Evaluation (2021.eacl-main)
Copied to clipboard
| Challenge: | Current manual evaluation methods for text summarization have low correlation with human judgements on summary quality. |
| Approach: | They conduct two evaluation experiments on two aspects of summaries’ linguistic quality (coherence and repetitiveness) they find that study parameters such as the total number of annotators and distribution of annotes to annotation items are often not fully reported. |
| Outcome: | The proposed methods can inflate type I errors up to eight-fold and the overall number of annotators can have a strong impact on study power. |
HelpSteer: Multi-attribute Helpfulness Dataset for SteerLM (2024.naacl-long)
Copied to clipboard
Zhilin Wang, Yi Dong, Jiaqi Zeng, Virginia Adams, Makesh Narsimhan Sreedhar, Daniel Egert, Olivier Delalleau, Jane Scowcroft, Neel Kant, Aidan Swope, Oleksii Kuchaiev
| Challenge: | Existing helpfulness preference datasets do not specify what makes some responses more helpful and others less helpful. |
| Approach: | They use a dataset that has annotated for correctness, coherence, complexity, and verbosity. |
| Outcome: | The dataset has annotations for correctness, coherence, complexity, and verbosity in addition to overall helpfulness of responses. |
Improving Generation and Evaluation of Visual Stories via Semantic Consistency (2021.naacl-main)
Copied to clipboard
| Challenge: | Story visualization is an underexplored task that requires a generative model to generate images . prior work has focused on image generation but there is room for improvement . |
| Approach: | They propose to add a dual learning framework to reinforce semantic alignment between story and generated images and a copy-transform mechanism to model sequentially-consistent story visualization. |
| Outcome: | The proposed models outperform text-to-image synthesis models on the story visualization task . the proposed models also improve visual quality, coherence and relevance . |
Think Beyond Words: Exploring Context-Relevant Visual Commonsense for Diverse Dialogue Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to generate intelligent open-domain dialogue agents only consider auxiliary commonsense stored in pure text, ignoring grounding information from the external visual world. |
| Approach: | They propose a VIsual Commonsense enhanced dialogue generaTOR that exploits auxiliary commonsense from images related to context to generate coherent and informative responses. |
| Outcome: | The proposed method outperforms the latest competitive methods in terms of coherence and diversity on two public datasets. |
A Disentangled Adversarial Neural Topic Model for Separating Opinions from Plots in User Reviews (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing topic models may extract topics associated with writers’ subjective opinions mixed with those related to factual descriptions. |
| Approach: | They propose a neural topic model combined with adversarial training to disentangle opinion topics from plot and neutral ones. |
| Outcome: | The proposed model shows improved coherence and variety of topics, consistent disentanglement rate, and superior sentiment classification performance to other supervised topic models. |
Guiding Abstractive Dialogue Summarization with Content Planning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for abstractive dialogue summarization struggle to maintain factual consistency between dialogue and summary. |
| Approach: | They propose a coarse-to-fine model for generating abstractive dialogue summaries and introduce a fact-aware reinforcement learning objective that improves the fact consistency between the dialogue and the generated summary. |
| Outcome: | The proposed model improves the quality of the generated summary, especially in coherence and consistency. |
Better Conversations by Modeling, Filtering, and Optimizing for Coherence and Diversity (D18-1)
Copied to clipboard
| Challenge: | Existing encoder-decoder models for open domain dialogue generate generic, uninformative, and non-coherent responses. |
| Approach: | They propose to introduce a measure of coherence as the GloVe embedding similarity between dialogue context and generated response to improve output diversity. |
| Outcome: | The proposed model improves on the OpenSubtitles corpus in terms of BLEU score and diversity metrics. |
Deductive Closure Training of Language Models for Coherence, Accuracy, and Updatability (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing language models (LMs) generate factually correct text and estimate truth values of individual claims, but they do not reflect a coherent, manipulable model of the world. |
| Approach: | They propose a method that uses language models to identify implications of (and contradictions within) the text they generate. |
| Outcome: | The proposed method improves LM factuality by 3-26% across the CREAK, MQuAKE, and Reversal Curse datasets. |
Human-in-the-loop Abstractive Dialogue Summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Abstractive dialogue summarization systems are trained to maximize the likelihood of human-written summaries, but there is still a huge gap in generating high-quality summary as determined by humans. |
| Approach: | They propose to incorporate different levels of human feedback into the training process . they ask humans to highlight salient information to be included in summaries . |
| Outcome: | The proposed model captures human-written summaries and compares them with state-of-the-art models on multiple datasets. |
Interpreting Topic Models in Byte-Pair Encoding Space (2025.coling-main)
Copied to clipboard
| Challenge: | Byte-pair encoding (BPE) is a popular method of tokenizing valid words onto a token space V b with a predetermined fixed size, and handling out-of-vocabulary words, breaking words into smaller tokens. |
| Approach: | They propose to interpret the recovery of valid words from these tokens as a ranking problem and apply existing evaluation measures to topic sets. |
| Outcome: | The proposed model interprets the recovery of valid words from these tokens as a ranking problem and applies existing evaluation measures. |
Is Summary Useful or Not? An Extrinsic Human Evaluation of Text Summaries on Downstream Tasks (2024.lrec-main)
Copied to clipboard
| Challenge: | a recent study focused on intrinsic evaluation, which assesses the quality of summaries, e.g. coherence, fluency, and informativeness, but it focused on task-based extrinsic evaluation to determine the usefulness of summarizations. |
| Approach: | They incorporate three downstream tasks to measure the usefulness of summaries . they find that fine-tuned models produce more useful summary across all three tasks . |
| Outcome: | The proposed model produces more useful summaries across all three tasks compared to zero-shot models . human evaluation provides more reliable performance assessment compared with automatic methods . |
Explicit Bayesian Inference to Uncover the Latent Themes of Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive generative capabilities, yet their inner mechanisms remain largely opaque. |
| Approach: | They propose a variational autoencoder-based neural topic model to interpret LLMs generation process through an explicit Bayesian framework by inferring latent topic variables via variational inference. |
| Outcome: | The proposed model outperforms state-of-the-art topic models on intrinsic measures of coherence and diversity on multiple datasets and shows significant gains on classification and summarization tasks. |