MQA-KEAL: Multi-hop Question Answering under Knowledge Editing for Arabic Language (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated significant capabilities across numerous application domains. |
| Approach: | They propose to use Multi-hop Questioning Answering under Knowledge Editing for Arabic Language to update and/or edit prior knowledge and test it via Multi-Hop Question Answering (MQA). |
| Outcome: | The proposed model outperforms baseline models by a significant margin . it can be used to update and/or edit prior knowledge and then test it with MQA . |
Similar Papers
PokeMQA: Programmable knowledge editing for Multi-hop Question Answering (2024.acl-long)
Copied to clipboard
| Challenge: | Multi-hop question answering (MQA) is one of the challenging tasks to evaluate machine’s comprehension and reasoning abilities, where large language models (LLMs) have widely achieved the human-comparable performance. |
| Approach: | They propose a framework to edit multi-hop question models to update model with up-to-date facts while avoiding expensive re-training or fine-tuning. |
| Outcome: | The proposed framework outperforms all competitors in multi-hop question answering tasks and consistently produces reliable reasoning process. |
MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for retraining from scratch are limited and only work on the recall of edited facts. |
| Approach: | They propose a benchmark method that allows users to ask multi-hop questions to assess whether edited models correctly answer questions where the answer should change as an entailed consequence of edited facts. |
| Outcome: | The proposed method outperforms existing models and scales well with LLMs (up to 175B) it is based on a memory-based approach that stores all edited facts externally while prompting the language model iteratively to generate answers consistent with the edited facts. |
LLM-Based Multi-Hop Question Answering with Knowledge Graph Integration in Evolving Environments (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for knowledge editing in Large Language Models face difficulties with multi-hop questions that require accurate fact identification and sequential logical reasoning. |
| Approach: | They propose a method that merges explicit knowledge representations of Knowledge Graphs with the linguistic flexibility of Large Language Models to convert free-form language into structured queries and fact triples. |
| Outcome: | The proposed method significantly surpasses state-of-the-art knowledge editing methods in the multi-hop question answering benchmark, MQuAKE. |
Interactive-KBQA: Multi-Turn Interactions for Knowledge Base Question Answering with Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Knowledge base question answering (KBQA) is a challenging task, particularly in parsing intricate questions into executable logical forms. |
| Approach: | They propose a framework to generate logical forms through direct interaction with knowledge bases (KBs) by annotating a dataset with step-wise reasoning processes. |
| Outcome: | The proposed framework achieves competitive results on the WebQuestionsSP, ComplexWebQuestIONS, KQA Pro, and MetaQA datasets with a minimal number of examples (shots). Importantly, the proposed model supports manual intervention, allowing for the iterative refinement of LLM outputs. |
ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic (2024.findings-acl)
Copied to clipboard
Fajri Koto, Haonan Li, Sara Shatnawi, Jad Doughman, Abdelrahman Sadallah, Aisha Alraeesi, Khalid Almubarak, Zaid Alyafeai, Neha Sengupta, Shady Shehata, Nizar Habash, Preslav Nakov, Timothy Baldwin
| Challenge: | evaluating language models in Arabic remains challenging due to limited datasets . focus has shift to reasoning and knowledge-intensive tasks due to lack of relevant datasets. |
| Approach: | They propose to use ArabicMMLU to evaluate models' understanding of Arabic . they use 40 tasks and 14,575 multiple-choice questions from school exams in different countries . |
| Outcome: | The ArabicMMLU is the first multi-task language understanding benchmark for the Arabic language . it is based on 40 tasks and 14,575 multiple-choice questions in modern standard Arabic . the models are based in different countries across North Africa, the Levant, and the Gulf regions . |
KELE: Residual Knowledge Erasure for Enhanced Multi-hop Reasoning in Knowledge Editing (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge editing techniques show limitations when applied to multi-hop reasoning . residual single-hop knowledge causes edited models to revert to original answers . |
| Approach: | They propose a knowledge editing method that incorporates a Knowledge Erasure mechanism for Large language model Editing (KELE) they propose an erasure function for residual knowledge and an injection function for new knowledge . |
| Outcome: | The proposed method significantly improves multi-hop reasoning capability of edited models. |
Uncertainty Guided Global Memory Improves Multi-Hop Question Answering (2023.emnlp-main)
Copied to clipboard
| Challenge: | Transformers are used to solve multi-hop question answering tasks that require reasoning over multiple parts of a long document. |
| Approach: | They propose a method that collects relevant information over the entire document and then combines it with local context to solve a multi-hop question answering task. |
| Outcome: | The proposed method improves on three MHQA datasets compared to the baseline model. |
MINTQA: A Multi-Hop Question Answering Benchmark for Evaluating LLMs on New and Long-tail Knowledge (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that large language models can handle knowledge with varying familiarity. |
| Approach: | They propose a benchmark to evaluate multi-hop question answering on new and tail knowledge . they use RAG to integrate external knowledge into large language models . |
| Outcome: | The proposed benchmark evaluates the multi-hop reasoning ability of large language models . it primarily evaluates their ability to handle knowledge with different levels of familiarity . |
Constraint-based Multi-hop Question Answering with Knowledge Graph (2022.naacl-industry)
Copied to clipboard
| Challenge: | Recent work addresses multi-hop KGQA, which requires reasoning across numerous edges of the KG. |
| Approach: | They propose to use KG embeddings to reduce KG sparsity by performing missing link prediction. |
| Outcome: | Empirical results show that the proposed method produces state-of-the-art results on three KGQA datasets. |
Robust Knowledge Editing via Explicit Reasoning Chains for Distractor-Resilient Multi-Hop QA (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models encode vast amounts of knowledge but remain static once trained, making timely integration of emerging facts prohibitively expensive via full retraining. |
| Approach: | They introduce a reasoning-chain-based editing framework that steers a pretrained LLM through four structured stages to filter distractors in a single pass. |
| Outcome: | The proposed framework steers a pretrained LLM through four structured stages to filter distractors in a single pass. |