GraphLoRA: Structure-Aware Low-Rank Adaptation for Large Language Model Recommendation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for translating collaborative information into textual prompts or injecting pre-trained embeddings into the LLM treat structural information as static input and fail to capture high-order relational dependencies. |
| Approach: | They propose a framework that generalizes low-rank adaptation from independent to structure-aware propagation by embedding a trainable graph message-passing network within the low-ranked adaptation pathway. |
| Outcome: | Experiments on multiple benchmarks show that GraphLoRA outperforms state-of-the-art recommendation methods and achieves superior generalization. |
Similar Papers
LeLoRA: Learnable Low-Rank Adaptation of Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to fine-tuning large language models (LLMs) rely on manually specified and fixed hyperparameters, resulting in suboptimal performance and low parameter efficiency. |
| Approach: | They propose a framework that allows for dynamically learned adaptive adaptation strategies to be used to fine-tune large language models. |
| Outcome: | The proposed framework outperforms baselines in adapting large language models. |
DenseLoRA: Dense Low-Rank Adaptation of Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Low-rank adaptation (LoRA) is an efficient approach for adapting large language models (LLMs) but many of the weights in these matrices are redundant, leading to inefficiencies in parameter utilization. |
| Approach: | They propose a low-rank adaptation approach that fine-tunes two low-ranked matrices and adapts them through a dense low-Rank matrix, improving parameter utilization and adaptation efficiency. |
| Outcome: | The proposed approach achieves 83.8% accuracy with only 0.01% of trainable parameters compared to LoRA's 80.8% with 0.70% of trainability parameters on LLaMA3-8B. |
TLoRA: Task-aware Low Rank Adaptation of Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing low-rank Adaptation (LoRA) methods address only one factor, often at the cost of increased training complexity or reduced practical efficiency. |
| Approach: | They propose a low-rank Adaptation framework that optimizes initialization and resource allocation at the outset of training. |
| Outcome: | The proposed framework performs excellently across various tasks while reducing the number of trainable parameters. |
CGBridge: Bridging Code Graphs and Large Language Models for Better Structure-Aware Code Understanding (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing structure-aware approaches treat structure as serialized text prompts or auxiliary training objectives, failing to provide explicit guidance during inference. |
| Approach: | They propose a plug-and-play method that enhances Large Language Models with Code Graph information through an external, trainable Bridge module. |
| Outcome: | The proposed method decouples structural reasoning from textual generation without updating the backbone. |
TableLoRA: Low-rank Adaptation on Table Structure Understanding for Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Tabular data are crucial in many fields and their understanding by large language models (LLMs) under high parameter efficiency paradigm is important. |
| Approach: | They propose a module that uses 2D LoRA to encode low-rank information on cell positions to improve table serialization and representation of two-dimensional structured information within a one-dimensional sequence. |
| Outcome: | Experiments on four tabular-related datasets show that TableLoRA outperforms vanilla LoRA and surpasses table encoding methods tested in control. |
PRILoRA: Pruned and Rank-Increasing Low-Rank Adaptation (2024.findings-eacl)
Copied to clipboard
| Challenge: | Several approaches to parameter-efficient fine-tuning have been proposed . low-rank Adaptation (LoRA) does not consider the varying importance of each layer . |
| Approach: | They propose a method that allocates a different rank for each layer and performs pruning throughout the training process. |
| Outcome: | The proposed method is based on eight GLUE benchmarks and is currently the state of the art. |
XRec: Large Language Models for Explainable Recommendation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Collaborative filtering (CF) is a widely adopted approach, but lacks the ability to provide explanations for the recommended items. |
| Approach: | They propose a model-agnostic framework that enables large language models to provide comprehensive explanations for user behaviors in recommender systems. |
| Outcome: | The proposed framework outperforms baseline approaches in explainable recommender systems. |
RanLoRA: Residual-aware Nonlinear Low-Rank Adaptation (2026.findings-acl)
Copied to clipboard
| Challenge: | Low-Rank Adaptation (LoRA) relying on linear low-rank projections restricts adaptation to linear subspaces, limiting flexibility on complex downstream tasks. |
| Approach: | They propose a nonlinear low-rank Adaptation approach that leverages pretrained weights to decompose them into principal components that are kept frozen and residual components that can be used for task-specific adaptation. |
| Outcome: | The proposed approach outperforms vanilla LoRA and representative variants on commonsense reasoning, image classification, and mathematical reasoning tasks. |
LLaSA: Large Language and Structured Data Assistant (2025.naacl-long)
Copied to clipboard
Yao Xu, Shizhu He, Jiabei Chen, ZengXiangrong ZengXiangrong, Bingning Wang, Guang Liu, Jun Zhao, Kang Liu
| Challenge: | Structured knowledge grounding (SKG) tasks are a key part of many NLP applications. |
| Approach: | They propose a framework for enhancing LLMs' ability to handle structured data . they represent various types of structured data in a unified hypergraph format . |
| Outcome: | The proposed framework outperforms existing methods on SKG tasks using LoRA finetuning. |
Enhancing High-order Interaction Awareness in LLM-based Recommender Model (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to model user-item interactions do not account for high-order interactions. |
| Approach: | They propose to enhance whole-word embeddings to enhance LLMs’ interpretation of graph-constructed interactions for recommendations without requiring graph pre-training. |
| Outcome: | The proposed model outperforms state-of-the-art methods in direct recommendations. |