Can Large Language Models Keep Up? Benchmarking Online Adaptation to Continual Knowledge Streams (2026.acl-long)
Copied to clipboard
Jiyeon Kim, Hyunji Lee, Dylan Zhou, Sue Hyun Park, Seunghyun Yoon, Trung Bui, Franck Dernoncourt, Sungmin Cha, Minjoon Seo
| Challenge: | Existing models and agentic memory systems fail to adapt robustly to OAKS, demonstrating delays in state-tracking and susceptibility to distraction within streaming environments. |
| Approach: | They propose a benchmark to evaluate models' ability to adapt to changing knowledge over streaming . they use two datasets to analyze how facts evolve over time . |
| Outcome: | The proposed benchmark evaluates models in an online adaptation setting over streaming, continually updating knowledge. |
Similar Papers
RAG or Learning? Understanding the Limits of LLM Adaptation under Continuous Knowledge Drift in the Real World (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to update or supplement large language models struggle under continuous knowledge drift. |
| Approach: | They propose a dynamic event benchmark and time-aware retrieval baseline that captures how knowledge evolves over time. |
| Outcome: | The proposed method enables systematic evaluation of model adaptation under continuous knowledge drift. |
EvoWiki: Evaluating LLMs on Evolving Knowledge (2025.acl-long)
Copied to clipboard
Wei Tang, Yixin Cao, Yang Deng, Jiahao Ying, Bo Wang, Yizhe Yang, Yuyue Zhao, Qi Zhang, Xuanjing Huang, Yu-Gang Jiang, Yong Liao
| Challenge: | Existing knowledge evolution benchmarks are static and fail to capture the evolving nature of LLMs and knowledge. |
| Approach: | They propose an evolving dataset that categorizes information into stable, evolved, and uncharted states. |
| Outcome: | The proposed dataset is auto-updatable and enables evaluation of continuously changing knowledge and newly released LLMs. |
TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models (2022.emnlp-main)
Copied to clipboard
Joel Jang, Seonghyeon Ye, Changho Lee, Sohee Yang, Joongbo Shin, Janghoon Han, Gyeonghun Kim, Minjoon Seo
| Challenge: | Language Models (LMs) become outdated as the world changes, a phenomenon called temporal misalignment. |
| Approach: | They propose a lifelong benchmark that utilizes the difference between consecutive snapshots of English Wikipedia and English Wikidata for training and evaluation. |
| Outcome: | The proposed benchmark can be trained on the difference between consecutive snapshots of English Wikipedia and English Wikidata for training and evaluation. |
LLM-Evolve: Evaluation for LLM’s Evolving Capability on Benchmarks (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks for large language models evaluate LLMs on i.i.d. tasks, overlooking their ability to learn iteratively from past experiences. |
| Approach: | They propose a framework which extends established benchmarks to sequential problem-solving settings and provides feedback after each round to build a demonstration memory that the models can query in future tasks. |
| Outcome: | The proposed framework can improve performance of LLMs by learning from past interactions and improve models' performance over time. |
Carpe diem: On the Evaluation of World Knowledge in Lifelong Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Current language models are trained on static data, implying that the encoded knowledge could go wrong as time passes. |
| Approach: | They propose a temporally evolving question-answering benchmark for language models . they use Wikipedia databases to test language models for dynamic knowledge in ever-changing world . |
| Outcome: | The proposed task aims to model the evolution-adaptability of language models in the real world. |
Beyond Static Toolsets: Self-Evolving LLM Tool Agents via Continual Documentation Adaptation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for evaluating tool usage assume static toolsets with fixed APIs and documentation. |
| Approach: | They propose a continual documentation adaptation framework that allows LLM agents to self-evolve by updating tool documentation. |
| Outcome: | The proposed framework improves performance on three evolution patterns on dynamic extensions of StableToolBench and RestBench. |
GrowOVER: How Can LLMs Adapt to Growing Real-World Knowledge? (2024.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge-based datasets are outdated due to the rapid evolution of knowledge. |
| Approach: | They propose a retrieval-interactive language model framework that evaluates and reflects on its answers for further re-retrieval. |
| Outcome: | The proposed framework performs comparably to or surpasses continuously trained language models. |
Continual Learning of Large Language Models (2025.emnlp-tutorials)
Copied to clipboard
| Challenge: | This tutorial explores the challenges of continual learning in large language models . participants will learn strategies to mitigate forgetting and manage data and evaluation pipelines . |
| Approach: | This tutorial offers a comprehensive exploration of continual learning in the context of large language models. |
| Outcome: | This tutorial explores the challenges of continual learning in large language models . participants will learn how to manage data and evaluation pipelines and adapt responsibly . |
From Static Inference to Dynamic Interaction: A Survey of Streaming Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing definitions of streaming LLMs are fragmented and lack a systematic taxonomy . large language models are pre-trained on static and full-context corpora . |
| Approach: | They propose a systematic taxonomy of current streaming Large Language Models and propose underlying methodologies for streaming LLMs. |
| Outcome: | The proposed model is based on data flow and dynamic interaction to clarify existing ambiguities. |
TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining (2025.acl-long)
Copied to clipboard
Jeffrey Li, Mohammadreza Armandpour, Seyed Iman Mirzadeh, Sachin Mehta, Vaishaal Shankar, Raviteja Vemulapalli, Samy Bengio, Oncel Tuzel, Mehrdad Farajtabar, Hadi Pouransari, Fartash Faghri
| Challenge: | Large language models (LLMs) trained on historical web data inevitably become outdated. |
| Approach: | They propose a web-scale dataset for time-continual pretraining of LLMs derived from 114 dumps of Common Crawl (CC) they also design time-stratified evaluations to assess how well various continual learning methods adapt to new data while retaining past knowledge. |
| Outcome: | The proposed benchmarks show that autoregressive meta-schedules combined with a fixed-ratio replay of older data can achieve comparable held-out loss to re-training from scratch, while requiring significantly less computation (2.6x). |