Mixture of Length and Pruning Experts for Knowledge Graphs Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing graph neural networks (GNNs) adopt rigid, query-agnostic path-exploration strategies limiting their ability to adapt to diverse linguistic contexts and semantic nuances. |
| Approach: | They propose a mixture-of-experts framework that personalizes path exploration . framework uses length experts that adaptively selects and weights candidate paths . it also uses pruning experts that evaluates candidate path from a complementary perspective . |
| Outcome: | The proposed framework shows superior performance on a diverse benchmark . it uses a mixture of experts that weights and selects path lengths according to query complexity . |
Similar Papers
Distance-Based Propagation for Efficient Knowledge Graph Reasoning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge graph completion (KGC) aims to predict unseen edges in knowledge graphs (KGs) . a few recent attempts to address this problem sacrifice the performance to gain efficiency. |
| Approach: | They propose a method that aggregates path information to solve this problem by aggregating paths in a fixed window for each source-target pair. |
| Outcome: | The proposed method can cut down on the number of propagated messages by 90% while achieving competitive performance on multiple KG datasets. |
Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent years have seen a surge of interest in improving the generation quality of commonsense reasoning tasks. |
| Approach: | They propose a method that diversifies the generative reasoning by a mixture of expert strategy on commonsense knowledge graphs to encourage various generation outputs. |
| Outcome: | The proposed method improves diversity while achieving on par performance on two GCR benchmarks, based on both automatic and human evaluations. |
HyperKGR: Knowledge Graph Reasoning in Hyperbolic Space with Graph Neural Network Encoding Symbolic Path (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for linking knowledge graphs are incomplete and rely on Euclidean embeddings . a hyperbolic GNN framework embeds recursive learning trees in hyperbolical space . |
| Approach: | They propose a hyperbolic GNN framework that embeds recursive learning trees in hyperbolical space and generates query-specific embeddings. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on multiple benchmark datasets. |
Bag of Tricks for Sparse Mixture-of-Experts: A Benchmark Across Reasoning, Efficiency, and Safety (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks focus on isolated aspects of MoE, with conflicting conclusions . a lack of consensus on optimal design choices is limiting to specific aspects of the model. |
| Approach: | They propose to evaluate two popular MoE backbones across four dimensions of design choices . they find token-level routing and z-loss regularization improve reasoning performance . |
| Outcome: | The proposed framework evaluates two popular MoE backbones on over eight metrics. |
KG-GPT: A General Framework for Reasoning on Knowledge Graphs Using Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Using large language models for complex reasoning tasks on knowledge graphs remains unexplored. |
| Approach: | They propose a multi-purpose framework leveraging large language models for complex reasoning tasks on knowledge graphs. |
| Outcome: | The proposed framework outperforms fully-supervised models in KG-based fact verification and KGQA benchmarks. |
Dropping Experts, Recombining Neurons: Retraining-Free Pruning for Sparse Mixture-of-Experts LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Sparse Mixture-of-Experts (SMoE) architectures require loading all expert parameters . previous work focused on expert pruning and merging but focused on neuron-level structure . |
| Approach: | They propose a task-agnostic framework for expert pruning and reconstruction . it prunes redundant experts using router statistics, then decomposes them into neuron-level expert segments . |
| Outcome: | The proposed framework reduces the number of experts and memory usage, making it easier to deploy. |
KARPA: A Training-free Method of Adapting Knowledge Graph as References for Large Language Model’s Reasoning Path Aggregation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for large language models (LLMs) are limited by step-by-step decision-making on KGs, or require fine-tuning or pre-training on specific KG. |
| Approach: | They propose a framework that harnesses the global planning abilities of large language models (LLMs) for efficient and accurate KG reasoning. |
| Outcome: | Extensive experiments show that the proposed framework achieves state-of-the-art performance in KGQA tasks, delivering both high efficiency and accuracy. |
Diversifying the Expert Knowledge for Task-Agnostic Pruning in Sparse Mixture-of-Experts (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have outstanding performance by learning a large number of model parameters on large amounts of data. |
| Approach: | They propose a method of grouping and pruning similar experts to improve the model’s parameter efficiency by a range of natural language tasks. |
| Outcome: | The proposed method outperforms other model pruning methods on a range of natural language tasks. |
TENP: Trapezoidal Expert Neuron Pruning For Mixture-of-Experts (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing compression approaches remove entire experts, disrupting routing topology and harming performance, or rely on unstructured weight pruning with limited practical efficiency. |
| Approach: | They propose a structured **T**rapezoidal **E**xpert **N**euron **P**running framework that uses a trapezoidal pattern to identify and retain important experts while applying expert neuron pruning (ENP) to less important experts. |
| Outcome: | The proposed framework outperforms the full-parameter model by 10% on code generation tasks under a sparse activation of experts and a 40% routing sparsity. |
Mixture-of-Partitions: Infusing Large Biomedical Knowledge Graphs into BERT (2021.emnlp-main)
Copied to clipboard
| Challenge: | Infusing factual knowledge into pre-trained models is fundamental for many knowledge-intensive tasks. |
| Approach: | They propose an infusion approach that partitions a large knowledge graph into smaller sub-graphs and infuses their specific knowledge into various BERT models using lightweight adapters. |
| Outcome: | The proposed approach improves the underlying BERTs and achieves new SOTA performance on six downstream tasks. |