| Challenge: | Existing language models rely on word correlation and are difficult to interpret . existing models often lack explicit representations for such information . |
| Approach: | They propose a memory-augmented approach to condition autoregressive language models on knowledge graphs. |
| Outcome: | The proposed model improves perplexity and bits per character in an autoregressive language model . it is complementary to token-based memory and enables causal interventions . |
Similar Papers
Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs (2024.findings-acl)
Copied to clipboard
Bowen Jin, Chulin Xie, Jiawei Zhang, Kashob Kumar Roy, Yu Zhang, Zheng Li, Ruirui Li, Xianfeng Tang, Suhang Wang, Yu Meng, Jiawei Han
| Challenge: | Existing studies suggest augmenting LLMs with external text corpora to alleviate hallucination problems. |
| Approach: | They propose to augment large language models with text units retrieved from external knowledge corpora to alleviate the issue. |
| Outcome: | The proposed framework outperforms baselines on GRBench with three LLMs and shows that iterative reasoning outperformed the baselines. |
More room for language: Investigating the effect of retrieval on language models (2024.naacl-short)
Copied to clipboard
| Challenge: | Retrieval-augmented language models are a promising alternative to standard pretraining, but little attention has been put into understanding what this type of training scheme does to the underlying language model when analyzed as a standalone -separated from the overall retrieval pipeline. |
| Approach: | They propose an ‘ideal retrieval’ methodology to study these models in a fully controllable setting and propose a retrieval augmentation methodology to examine their effects. |
| Outcome: | The proposed model saves substantially less world knowledge in their weights, but is worse at comprehending global context. |
Learning beyond Datasets: Knowledge Graph Augmented Neural Networks for Natural Language Processing (N18-1)
Copied to clipboard
| Challenge: | Currently, machine learning is limited in scalability and is limited to specific training data. |
| Approach: | They propose to enhance learning models with world knowledge in the form of Knowledge Graph fact triples for natural language processing tasks. |
| Outcome: | The proposed method is highly scalable to the amount of prior information that has to be processed and can be applied to any generic NLP task. |
Graph-Assisted Large Language Models: A Perspective on Mitigating Intrinsic Limitations (2026.findings-acl)
Copied to clipboard
Haitong Luo, Fali Wang, Weiyao Zhang, Xianren Zhang, Zhiwei Zhang, Tianxiang Zhao, Minhua Lin, Jiahao Zhang, Hui Liu, Xianfeng Tang, Qi He, Suhang Wang, Xuying Meng, Yujun Zhang
| Challenge: | Large language models exhibit intrinsic limitations such as knowledge cutoff, single-threaded reasoning that hinders finer-grained branch and aggregation, and rigid collaboration mechanisms that struggle to coordinate specialized capabilities. |
| Approach: | They propose a taxonomy spanning *Graph-Assisted Knowledge Augmentation*, *Graph Assisted Reasoning and Planning*, and *Graphed LLM Collaboration*. |
| Outcome: | The proposed models show that graphs can augment and correct LLMs and support dynamic coordination among experts and agents in collaborative settings. |
Retrieving Multimodal Information for Augmented Generation: A Survey (2023.findings-emnlp)
Copied to clipboard
Ruochen Zhao, Hailin Chen, Weishi Wang, Fangkai Jiao, Xuan Long Do, Chengwei Qin, Bosheng Ding, Xiaobao Guo, Minzhi Li, Xingxuan Li, Shafiq Joty
| Challenge: | Large Language Models (LLMs) are increasingly using multimodality to augment their generation ability, but there is no unified perception of at which stage and how to incorporate different modalities. |
| Approach: | They propose to use multimodality to augment Large Language Models (LLMs) this will provide scholars with a deeper understanding of the methods' applications and encourage them to adapt existing techniques to the fast-growing field of LLMs. |
| Outcome: | The proposed methods improve factuality, reasoning, interpretability, and robustness of the generated content. |
Reimagining Retrieval Augmented Language Models for Answering Queries (2023.findings-acl)
Copied to clipboard
Wang-Chiew Tan, Yuliang Li, Pedro Rodriguez, Richard James, Xi Victoria Lin, Alon Halevy, Wen-tau Yih
| Challenge: | Large language models (LLMs) are expensive to train, deploy, and maintain, both financially and in terms of environmental impact. |
| Approach: | They present a reality check on large language models and compare their predictions to retrieval-augmented language models. |
| Outcome: | The proposed models fare better on question answering tasks and have become the foundation of impressive demos like Chat-GPT. |
Query-Aware Graph Attention for Precise Subgraph Retrieval in Knowledge-Augmented Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing Knowledge Graph (KG)-based Retrieval-Augmented Generation (RAG) systems insufficiently model the interaction between query semantics and relation types, resulting in imprecise subgraph retrieval and unstable reasoning. |
| Approach: | They propose a retrieval framework that integrates query semantics and relation embeddings directly into the attention mechanism. |
| Outcome: | Experiments on WebQSP and CWQ establish new state-of-the-art results in both Triple Recall and Answer Recall. |
Memory-Augmented LLM-based Multi-Agent System for Automated Feature Generation on Tabular Data (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for automated feature generation rely on predefined operator libraries and do not incorporate feature semantics, limiting their ability to produce high-quality features. |
| Approach: | They propose a Memory-Augmented LLM-based Multi-Agent System (MALMAS) that decomposes the generation process into agents with distinct responsibilities. |
| Outcome: | The proposed method extracts informative features from raw tabular data without manual intervention and is crucial for accurate, generalizable machine learning. |
Prediction-Augmented Generation for Automatic Diagnosis Tasks (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) adopt autoregressive architecture, predicting the next word token based on the preceding context. |
| Approach: | They propose a method that integrates task-specific predictive models as external tools to improve model generation quality and accuracy. |
| Outcome: | The proposed method improves the generation quality and predictive accuracy of large language models in inference-driven tasks. |
Knowledge-augmented Financial Market Analysis and Report Generation (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods to generate financial market analysis text require extensive financial knowledge and skill of financial analysts. |
| Approach: | They propose a task to generate financial market analysis reports using financial market data using a financial knowledge graph. |
| Outcome: | The proposed framework outperforms large-scale language models and retrieval-augmented baselines in the financial market analysis generation task. |