Unveiling Attractor Cycles in Large Language Models: A Dynamical Systems View of Successive Paraphrasing (2025.acl-long)
Copied to clipboard
| Challenge: | Dynamical systems theory provides a framework for understanding iterative processes and evolution over time. |
| Approach: | They propose to apply this perspective to large language models which iteratively map input text to output text and re-express meaning with linguistic variation. |
| Outcome: | The proposed model reveals that paraphrases re-express meaning with linguistic variation limiting linguistic diversity . |
Similar Papers
How Do Large Language Models Perform in Dynamical System Modeling (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent data-driven methods often use graph neural networks (GNNs) to learn interactions between objects. |
| Approach: | They propose prompting techniques for dynamical system modeling and evaluate their performance . they find that large language models demonstrate competitive performance without training . |
| Outcome: | The proposed methods show competitive performance without training compared to state-of-the-art methods in dynamical system modeling. |
Exploring Attention Attractors in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies have suggested that attention attractors function as "summary tokens" while others speculate that tokens with weaker semantics attract high attention, they act as attention sinks that offload excessive attention. |
| Approach: | They examine attention attractors, tokens that draw significantly high attention, in large language models. |
| Outcome: | The proposed models are able to capture long-range dependencies within a given context. |
Continual Learning of Large Language Models (2025.emnlp-tutorials)
Copied to clipboard
| Challenge: | This tutorial explores the challenges of continual learning in large language models . participants will learn strategies to mitigate forgetting and manage data and evaluation pipelines . |
| Approach: | This tutorial offers a comprehensive exploration of continual learning in the context of large language models. |
| Outcome: | This tutorial explores the challenges of continual learning in large language models . participants will learn how to manage data and evaluation pipelines and adapt responsibly . |
LLMs learn governing principles of dynamical systems, revealing an in-context neural scaling law (2024.emnlp-main)
Copied to clipboard
| Challenge: | LLaMA-2 language model is capable of in-context time series extrapolation without specific prompting or fine-tuning, revealing an in-constitution version of a neural scaling law. |
| Approach: | They propose an algorithm for extracting probability density functions of multi-digit numbers directly from Large language models (LLMs) LLaMA-2 is a language model trained on text and can extrapolate dynamical system time series without prompting or engineering . |
| Outcome: | The proposed model can extrapolate dynamical systems without prompting or engineering . it also achieves an in-context version of a neural scaling law . |
Repeated Sequences Reveal Gaps between Large Language Models and Natural Language (2026.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation methods provide limited insight into the long-range organization of generated text. |
| Approach: | They propose a framework for evaluation based on repeatedsubsequences . they compare their distribution across scales and their results to Rényi entropies . |
| Outcome: | The proposed framework relates distribution of results to higher-order Rényi entropies on human-written and length-matched GPT-generated texts. |
Are Large Language Model Temporally Grounded? (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent large language models lack a consistent temporal model of textual narratives . sentence ordering in unlabelled texts is only weakly correlated with event ordering . |
| Approach: | They evaluate LLMs with textual narratives and evaluate their common-sense knowledge . they find that LLM models struggle the most with self-consistency . |
| Outcome: | The proposed models lack a consistent temporal model of textual narratives. |
Temporal Flattening in LLM-Generated Text: Comparing Human and LLM Writing Trajectories (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly used in daily applications, from content generation to code writing. |
| Approach: | They construct a longitudinal dataset of 412 human authors and 6,086 documents spanning 2012–2024 and compare them to trajectories generated by three representative LLMs. |
| Outcome: | The results show that LLMs produce greater lexical diversity but exhibit substantially reduced semantic and cognitive–emotional drift relative to humans. |
A Survey of Post-Training Scaling in Large Language Models (2025.acl-long)
Copied to clipboard
Hanyu Lai, Xiao Liu, Junjie Gao, Jiale Cheng, Zehan Qi, Yifan Xu, Shuntian Yao, Dan Zhang, Jinhua Du, Zhenyu Hou, Xin Lv, Minlie Huang, Yuxiao Dong, Jie Tang
| Challenge: | Large language models (LLMs) have demonstrated proficiency in understanding and generating human natural languages. |
| Approach: | They propose a framework for scaling large language models using supervised fine-tuning, RLxF and test-time compute methodologies. |
| Outcome: | The proposed model can be used to understand and generate human natural languages. |
Demystifying Verbatim Memorization in Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that Large Language Models (LLMs) memorize long sequences verbatim, with serious copyright and privacy implications. |
| Approach: | They develop a framework to study verbatim memorization in a controlled setting by continuing pre-training from Pythia checkpoints with injected sequences. |
| Outcome: | The proposed framework creates a control model M () and a treatment model M with injected sequences. |
Rethinking Long Context Generation from the Continual Learning Perspective (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) struggle with processing long contexts due to the limited context window. |
| Approach: | They propose to combine a limited context window with a continual learning perspective to improve LLMs' efficiency in processing long contexts. |
| Outcome: | The proposed models improve the performance of Large Language Models (LLMs) by integrating learning strategies with existing approaches. |