FragRel: Exploiting Fragment-level Relations in the External Memory of Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to process contexts with unlimited length are limited to finite expansion length or prone to performance degradation when dealing with very long contexts. |
| Approach: | They propose to exploit fragment-level relations in external memory to hierarchically process the long text. |
| Outcome: | The proposed model improves story understanding, repository-level code generation, and long-term chatting. |
Similar Papers
LLM×MapReduce: Simplified Long-Sequence Processing using Large Language Models (2025.acl-long)
Copied to clipboard
Zihan Zhou, Chong Li, Xinyi Chen, Shuo Wang, Yu Chao, Zhili Li, Haoyu Wang, Qi Shi, Zhixing Tan, Xu Han, Xiaodong Shi, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing studies have focused on extending the context length of large language models (LLMs) due to their quadratic computational complexity and a lack of high-quality long training examples, most LLMs are trained with a limited window size. |
| Approach: | They propose a training-free framework that enables large language models to effectively process long texts using a divide-and-conquer strategy for comprehensive document understanding. |
| Outcome: | The proposed framework outperforms open-source and commercial long-context LLMs and is compatible with several models. |
Dynamic Chunking and Selection for Reading Comprehension of Ultra-Long Context in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Current methods for improving large language models rely on splitting long contexts into fixed-length chunks, compromising accuracy. |
| Approach: | They propose a method for dynamically separating and selecting chunks of long context, facilitating a more streamlined input for LLMs. |
| Outcome: | The proposed approach outperforms baseline methods on single-hop and multi-hop question-answering benchmarks. |
How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advances (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are impressive in solving tasks, but they can quickly be outdated after deployment. |
| Approach: | They provide a review of recent advances in aligning deployed large language models with the ever-changing world knowledge. |
| Outcome: | The proposed models can be used to perform various tasks directly through in-context learning or for further fine-tuning for domain-specific uses. |
Flexibly Utilize Memory for Long-Term Conversation via a Fragment-then-Compose Framework (2025.emnlp-main)
Copied to clipboard
Cai Ke, Yiming Du, Bin Liang, Yifan Xiang, Lin Gui, Zhongyang Li, Baojun Wang, Yue Yu, Hui Wang, Kam-Fai Wong, Ruifeng Xu
| Challenge: | Large language models extract useful information from conversation history to enhance the response in long-term conversations. |
| Approach: | They propose a Fragment-then-Compose framework to optimize memory utilization for long-term open-domain conversation. |
| Outcome: | The proposed framework can be used to extract useful information from conversation history . it can be adapted to different situations and improve response generation . |
Semi-automatic Data Enhancement for Document-Level Relation Extraction with Distant Supervision from Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Document-level Relation Extraction (DocRE) is a task that aims to extract relations from a long context. |
| Approach: | They propose an automated annotation method that integrates an LLM and a natural language inference module to generate relation triples. |
| Outcome: | The proposed method can extract relations from document-level relation datasets with minimal human effort. |
Out-of-Context Reasoning in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a lightweight technique trains only new token embeddings on axioms and evaluates them on unseen tasks. |
| Approach: | They propose a lightweight technique that trains only new token embeddings on axioms . they train only new embeddables and evaluate them on unseen tasks . |
| Outcome: | The proposed technique trains only new token embeddings on axioms and evaluates them on unseen tasks. |
Evaluating Very Long-Term Conversational Memory of LLM Agents (2024.acl-long)
Copied to clipboard
| Challenge: | Existing studies on long-term open-domain dialogues focus on evaluating responses within contexts spanning no more than five chat sessions. |
| Approach: | They propose a machine-human pipeline to generate very long-term dialogues by leveraging LLMs and retrieval augmented generation techniques. |
| Outcome: | The proposed pipeline generates very long-term dialogues using LLMs and RAGs . the generated conversations are verified and edited by human annotators for long-range consistency and grounding to the event graphs. |
EpMAN: Episodic Memory AttentioN for Generalizing to Longer Contexts (2025.acl-long)
Copied to clipboard
Subhajit Chaudhury, Payel Das, Sarathkrishna Swaminathan, Georgios Kollias, Elliot Nelson, Khushbu Pahwa, Tejaswini Pedapati, Igor Melnyk, Matthew Riemer
| Challenge: | Recent advances in Large Language Models (LLMs) have yielded impressive successes on many language tasks, but efficient processing of long contexts remains a significant challenge. |
| Approach: | They propose a method for processing long contexts in an episodic memory module while holistically attending to semantically-relevant context chunks. |
| Outcome: | The proposed method outperforms baseline decoders on multiple long-context recall and question-answering benchmarks on 16k to 256k tokens. |
SEGMENT+: Long Text Processing with Short-Context Language Models (2024.emnlp-main)
Copied to clipboard
Wei Shi, Shuang Li, Kerun Yu, Jinglei Chen, Zujie Liang, Xinhui Wu, Yuxi Qian, Feng Wei, Bo Zheng, Jiaqing Liang, Jiangjie Chen, Yanghua Xiao
| Challenge: | Existing frameworks that increase context window do not guarantee robust performance across long input tasks. |
| Approach: | They propose a framework that enables language models to handle extended inputs within limited context windows efficiently. |
| Outcome: | The framework improves performance on long-document question-answering and Needle-in-a-Haystack tasks. |
LlmLink: Dual LLMs for Dynamic Entity Linking on Long Narratives with Collaborative Memorisation and Prompt Optimisation (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods focus on supervised fine-tuning or limited to one-off prediction, which poses a challenge where the context is long. |
| Approach: | They propose a dynamic approach to CoREFerence resolution in chunked long narratives by deploying dual Large Language Models. |
| Outcome: | The proposed model achieves performance gains over existing models and fine-tuning approaches on long narrative datasets, significantly reducing the resources required for inference and training. |