HAWK: Highlighting Entity-aware Knowledge for Alleviating Information Sparsity in Long Contexts (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a problem of information sparsity in QA tasks is causing fragmentation of textual data . highlighting entity-AWare Knowledge (HAWK) framework can be used to address this problem . |
| Approach: | a framework is proposed to highlight key information in a context and structuralize it in an entity-aware manner. |
| Outcome: | a proposed framework improves QA tasks with long contexts by highlighting key information in a context . the framework achieves a 27.6-point F1 score increase and an average win rate of 76.75% . |
Similar Papers
SEARCH-R: Structured Entity-Aware Retrieval with Chain-of-Reasoning Navigator for Multi-hop Question Answering (2026.findings-acl)
Copied to clipboard
FU Yuqing, Yimin Deng, Wanyu Wang, Yuhao Wang, Yejing Wang, Hongshi Liu, Yiqi Wang, Xiao Han, Maolin Wang, Guoshuai Zhao, Yi Chang, Xiangyu Zhao
| Challenge: | Existing approaches to multi-hop question answering lack effective control over reasoning paths, leading to astray results. |
| Approach: | They propose a framework for multi-hop question answering that trains an end-to-end reasoning path navigator to provide a powerful sub-question decomposer by fine-tuning the Llama3.1-8B model. |
| Outcome: | The proposed framework trains an end-to-end reasoning path navigator . it is able to provide a powerful sub-question decomposer by fine-tuning the Llama3.1-8B model . |
Search-in-Context: Efficient Multi-Hop QA over Long Contexts via Monte Carlo Tree Search with Dynamic KV Retrieval (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to multihop question answering (MHQA) over long contexts are often neglecting explicit reasoning or incurring expensive computational costs due to full-attention mechanisms over long contextuals. |
| Approach: | They propose a framework that integrates Monte Carlo Tree Search (MCTS) with dynamic key-value retrieval to enable iterative, context-aware reasoning. |
| Outcome: | The proposed framework integrates Monte Carlo Tree Search (MCTS) with dynamic key-value (KV) retrieval to enable iterative, context-aware reasoning. |
AttnComp: Attention-Guided Adaptive Context Compression for Retrieval-Augmented Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for augmented large language models suffer from irrelevant retrieved content . existing methods struggle to adapt compression rates for different context, maintain low latency . |
| Approach: | We propose an adaptive, efficient and context-aware compression framework to reduce retrieved content . AttnComp uses a top-p compression algorithm to retain the minimal set of documents whose attention weights exceed a threshold. |
| Outcome: | Experiments show that AttnComp outperforms existing compression methods and uncompressed baselines in achieving higher accuracy with substantial compression rates and lower latency. |
Can’t Remember Details in Long Documents? You Need Some R&R (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Long-context large language models miss important information in the middle of context documents . a recent study shows that LLMs can be used for document-based QA tasks . |
| Approach: | They propose a prompt-based method called *reprompting* and *in-context retrieval* to alleviate this effect in document-based QA. |
| Outcome: | The proposed method improves QA accuracy on documents up to 80k tokens in length. |
From Logical to Computational Sparsity: Structure-Aware Block-Sparse Attention for Long-Code Completion (2026.acl-long)
Copied to clipboard
| Challenge: | Existing sparse attention methods for long-context generation pose high latency . general sparsity methods cause excessive accuracy degradation without considering code structure . |
| Approach: | They propose a training-free **S**tructure-**a**ware **b**lock-spa**r**s**e** attention mechanism that bridges the gap between logical and computational sparsity. |
| Outcome: | The proposed method reduces TTFT by 45-55% while maintaining accuracy within 3% of dense attention. |
Tailoring Memory Granularity for Multi-Hop Reasoning over Long Contexts (2026.findings-eacl)
Copied to clipboard
| Challenge: | Extensive experiments on long-context multi-hop question answering benchmarks show TAG achieves state-of-the-art performance. |
| Approach: | They propose a framework that prestructures memory into diverse granularities and employs a reward-guided navigator to adaptively compose hybrid memory tailored to each query. |
| Outcome: | Experiments on long-context multi-hop question answering show that the framework achieves state-of-the-art performance. |
KV Cache Compression, But What Must We Give in Return? A Comprehensive Benchmark of Long Context Capable Approaches (2024.findings-emnlp)
Copied to clipboard
Jiayi Yuan, Hongyi Liu, Shaochen Zhong, Yu-Neng Chuang, Songchen Li, Guanchu Wang, Duy Le, Hongye Jin, Vipin Chaudhary, Zhaozhuo Xu, Zirui Liu, Xia Hu
| Challenge: | Long context capability is a crucial competency for large language models as it mitigates the human struggle to digest long-form texts. |
| Approach: | They propose to evaluate 10+ state-of-the-art approaches for long context-capable LLMs. |
| Outcome: | The proposed methods are compared against 10+ state-of-the-art approaches across seven categories of long context tasks. |
Leveraging Contextual Information for Effective Entity Salience Detection (2024.findings-naacl)
Copied to clipboard
Rajarshi Bhowmik, Marco Ponza, Atharva Tendle, Anant Gupta, Rebecca Jiang, Xingyu Lu, Qian Zhao, Daniel Preotiuc-Pietro
| Challenge: | Prior work on salient entity detection focused on machine learning models that require heavy feature engineering. |
| Approach: | They propose to fine-tune medium-sized language models with a cross-encoder style architecture to achieve significant performance gains over feature engineering approaches. |
| Outcome: | The proposed model fine-tunes medium-sized pre-trained language models with a cross-encoder style architecture yields substantial performance gains over feature engineering approaches. |
KV Pareto: Systems-Level Optimization of KV Cache and Model Compression for Long Context Inference (2026.eacl-industry)
Copied to clipboard
| Challenge: | Long-context Large Language Models (LLMs) face significant memory bottlenecks due to the linear growth of key-value (KV) cache with sequence length. |
| Approach: | They propose a framework that maps the trade-off frontier between total memory consumption and task accuracy across three complementary optimization techniques. |
| Outcome: | The proposed model-specific configurations achieve 68-78% total memory reduction with minimal (1-3%) accuracy degradation on long-context tasks. |
COMPASS: Enhancing Agent Long-Horizon Reasoning with Evolving Context (2026.acl-long)
Copied to clipboard
| Challenge: | Long-horizon tasks that require sustained reasoning and multiple tool interactions remain challenging for LLM agents. |
| Approach: | They propose a framework that separates tactical execution, strategic oversight, and context organization into three specialized components. |
| Outcome: | The proposed framework improves accuracy by 20% relative to baselines on GAIA, BrowseComp, and Humanity’s Last Exam tasks. |