Information Integration in Large Language Models is Gated by Linguistic Structural Markers (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that LLMs can integrate information across multiple scales of language . |
| Approach: | They propose to quantify the information integration window of large language models and measure the information loss from relying solely on the local window. |
| Outcome: | The proposed method shows that the integration window of large language models is strongly modulated by sentence boundaries. |
Similar Papers
Do Large Language Models Know How Much They Know? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models are highly capable systems, but their capabilities and limitations are unclear. |
| Approach: | They develop a benchmark that challenges LLMs to recall all information they possess on specific topics. |
| Outcome: | The proposed model can recall excessive, insufficient, or the precise amount of information they possess on a given topic, indicating their awareness of how much they know about the given topic. |
Lost in the Distance: Large Language Models Struggle to Capture Long-Distance Relational Knowledge (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent large language models have demonstrated impressive capabilities in handling long contexts . however, as context length increases, LLMs struggle more with filtering out irrelevant information . |
| Approach: | They propose to use unrelated sentences to capture relational knowledge over long contexts . they find that LLMs can handle edge noise with little impact, but can reason about distant relationships . |
| Outcome: | The proposed model can handle edge noise with little impact, but its ability to reason about distant relationships declines as the noise grows. |
Working Memory Identifies Reasoning Limits in Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Using large language models, we examine the limitations of their cognitive capabilities and their working memory. |
| Approach: | They examine the limitations of large language models from a scaling perspective . they also assess various prompting strategies, revealing their diverse impacts on LLM performance. |
| Outcome: | The proposed models perform poorly on n-back tasks and on prompting strategies. |
Knowledge Boundary of Large Language Models: A Survey (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) store vast amount of knowledge in their parameters, but they still have limitations in the memorization and utilization of certain knowledge. |
| Approach: | They propose a comprehensive definition of the LLM knowledge boundary and introduce a formalized taxonomy categorizing knowledge into four distinct types. |
| Outcome: | The proposed definition of the LLM knowledge boundary and taxonomy categorizes knowledge into four distinct types . aims to offer a comprehensive overview, facilitate access to key issues, and inspire further advancements in LLM research. |
When Large Language Models Meet Speech: A Survey on Integration Approaches (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models have spurred interest in expanding their application beyond text-based tasks. |
| Approach: | They propose to categorize the integration of speech with LLMs into three main approaches . they demonstrate how these methods are applied across various speech-related applications . |
| Outcome: | The proposed methods are applied across speech-related applications and highlight the challenges in this field to offer inspiration for future research. |
Can Large Language Models Understand Context? (2024.findings-eacl)
Copied to clipboard
Yilun Zhu, Joel Moniz, Shruti Bhargava, Jiarui Lu, Dhivya Piraviperumal, Site Li, Yuan Zhang, Hong Yu, Bo-Hsiang Tseng
| Challenge: | Existing evaluation methodologies for Large Language Models (LLMs) have been inadequate to evaluate their ability to understand contextual features. |
| Approach: | They propose a benchmark to assess large language models' ability to understand context by adapting existing datasets to suit their evaluation. |
| Outcome: | The proposed model performs better under the in-context learning pretraining scenario than state-of-the-art models. |
Think in Sentences: Explicit Sentence Boundaries Enhance Language Model’s Capabilities (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies focus on dummy tokens but fail to leverage the inherent sentence-level structure of natural language. |
| Approach: | They propose a method that inserts delimiters at sentence boundaries to enhance large language models' capabilities. |
| Outcome: | The proposed method improves performance on 7B LLMs to 600B Deepseek-V3 with 7.7% gains on GSM8k and 12.5% on DROP. |
Extracting Linguistic Information from Large Language Models: Syntactic Relations and Derivational Knowledge (2025.emnlp-main)
Copied to clipboard
| Challenge: | Using large language models, we study their morphosyntactic competence and generalization capabilities. |
| Approach: | They propose to use morphosyntactic tasks to study their linguistic knowledge and generalization capabilities to extract different types of morphological structure for typologically diverse languages. |
| Outcome: | The proposed models outperform GPT-4o and LLaMA 3.3-70B in all diagnostic tasks, but show little evidence of abstract morphological rule learning. |
Are Large Vision Language Models up to the Challenge of Chart Comprehension and Reasoning (2024.findings-emnlp)
Copied to clipboard
Mohammed Saidul Islam, Raian Rahman, Ahmed Masry, Md Tahmid Rahman Laskar, Mir Tafseer Nayeem, Enamul Hoque
| Challenge: | Recent studies have demonstrated that large vision language models (LVLMs) are not multi-modal and lack multi-tasking capabilities. |
| Approach: | They evaluate the performance of large vision language models (LVLMs) for chart understanding and reasoning tasks and compare them to open-source models. |
| Outcome: | The proposed models demonstrate impressive abilities in generating fluent texts covering high-level data insights, but they also encounter common problems like hallucinations, factual errors, and data bias. |
Scaling in Cognitive Modelling: a Multilingual Approach to Human Reading Times (2023.acl-short)
Copied to clipboard
| Challenge: | Neural language models provide conditional probability distributions over the lexicon that are predictive of human processing times. |
| Approach: | They propose to use a transformer-based model to generate probabilistic estimates that are less predictive of early eye-tracking measurements reflecting lexical access and early semantic integration. |
| Outcome: | The proposed models show that larger models capture late eye-tracking measurements that reflect the full integration of a word into the current language context. |