Papers with GPT-J
RedCoast: A Lightweight Tool to Automate Distributed Training of LLMs on Any GPU/TPUs (2024.naacl-demo)
Copied to clipboard
| Challenge: | Recent advances in machine learning (ML) are attributed to large language models (LLMs), but their escalating memory requirements require developers to partition a large model to distribute it across multiple GPUs or TPUs. |
| Approach: | They propose a lightweight and user-friendly tool to automate distributed training and inference for LLMs and to simplify ML pipeline development. |
| Outcome: | The proposed tool automates distributed training and inference for LLMs, and simplifies ML pipeline development. |
Can We Edit Factual Knowledge by In-Context Learning? (2023.emnlp-main)
Copied to clipboard
| Challenge: | In-context knowledge editing (IKE) is a new paradigm for NLP research that can be applied to large language models with tens or hundreds of parameters. |
| Approach: | They propose to use in-context knowledge editing (IKE) without gradient updating to edit factual knowledge without a gradient update. |
| Outcome: | The proposed method achieves a competitive success rate compared to gradient-based methods on GPT-J but with fewer side effects. |
CoME: An Unlearning-based Approach to Conflict-free Model Editing (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) often retain outdated or incorrect information from pre-training, which undermines their reliability. |
| Approach: | They propose a conflict-free model editing framework that selectively removes outdated knowledge from LLMs to improve their accuracy and reliability. |
| Outcome: | The proposed framework improves both editing accuracy and model reliability when applied to existing editing methods. |
Lifelong Model Editing with Graph-Based External Memory (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for post-training model editing suffer from overfitting and catastrophic forgetting. |
| Approach: | They propose a framework that leverages hyperbolic geometry and graph neural networks for precise and stable model edits. |
| Outcome: | Experiments on CounterFact, CounterFACT+, and MQuAKE with GPT2-XL and GPT-J show that HYPE significantly enhances edit stability, factual accuracy, and multi-hop reasoning. |
KELE: Residual Knowledge Erasure for Enhanced Multi-hop Reasoning in Knowledge Editing (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge editing techniques show limitations when applied to multi-hop reasoning . residual single-hop knowledge causes edited models to revert to original answers . |
| Approach: | They propose a knowledge editing method that incorporates a Knowledge Erasure mechanism for Large language model Editing (KELE) they propose an erasure function for residual knowledge and an injection function for new knowledge . |
| Outcome: | The proposed method significantly improves multi-hop reasoning capability of edited models. |