Retrieval Augmented Instruction Tuning for Open NER with Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies have focused on integrating large language models (LLMs) with information extraction (IE) however, the best approach to incorporate information with LLMs for IE remains an open question. |
| Approach: | They propose to use a Chinese IT dataset to perform RA-IT for IE . they use semantically similar examples from the training dataset as the context . |
| Outcome: | The proposed approach is evaluated in English and Chinese scenarios. |
Similar Papers
INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning (2024.acl-long)
Copied to clipboard
Yutao Zhu, Peitian Zhang, Chenghao Zhang, Yifei Chen, Binyu Xie, Zheng Liu, Ji-Rong Wen, Zhicheng Dou
| Challenge: | Large language models (LLMs) have demonstrated impressive capabilities in various natural language processing tasks, but their application to information retrieval tasks is still challenging due to the infrequent occurrence of many IR-specific concepts in natural language. |
| Approach: | They propose to use instruction tuning to enhance LLMs' proficiency in IR tasks by combining a dataset with manually written templates to analyze the effects of instruction design, template diversity, few-shot demonstrations, and the volume of instructions. |
| Outcome: | The proposed model can be used to perform query understanding, document understanding, and query-document relationship understanding tasks. |
RaLLe: A Framework for Developing and Evaluating Retrieval-Augmented Large Language Models (2023.emnlp-demo)
Copied to clipboard
Yasuto Hoshi, Daisuke Miyashita, Youyang Ng, Kento Tatsuno, Yasuhiro Morioka, Osamu Torii, Jun Deguchi
| Challenge: | Existing libraries for building R-LLMs provide high-level abstractions without sufficient transparency for evaluating and optimizing prompts within specific inference processes. |
| Approach: | They propose an open-source framework to facilitate the development, evaluation, and optimization of R-LLMs for knowledge-intensive tasks. |
| Outcome: | The framework improves hand-crafted prompts, inference processes and quantitatively measures overall system performance. |
Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) encapsulate a vast amount of factual information within their pre-trained weights. |
| Approach: | They compare unsupervised fine-tuning and retrieval-augmented generation approaches to learning new factual information. |
| Outcome: | The proposed models outperform unsupervised fine-tuning and retrieval-augmented generation (RAG) on knowledge-intensive tasks across different topics. |
Retrieval-Augmented Retrieval: Large Language Models are Strong Zero-Shot Retriever (2024.findings-acl)
Copied to clipboard
Tao Shen, Guodong Long, Xiubo Geng, Chongyang Tao, Yibin Lei, Tianyi Zhou, Michael Blumenstein, Daxin Jiang
| Challenge: | Large-scale retrieval is indispensable in information-seeking tasks such as open-domain question answering and knowledgegrounded dialogue. |
| Approach: | They propose to use a large language model (LLM) to augment a query with its potential answers by prompting LLMs with a composition of the query and the query’s in-domain candidates. |
| Outcome: | The proposed method breaks brute-force combinations of retrievers with LLMs and lifts the performance of zero-shot retrieval to be very competitive on benchmark datasets. |
Improving Few-Shot Cross-Domain Named Entity Recognition by Instruction Tuning a Word-Embedding based Retrieval Augmented Large Language Model (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing approaches to named entity recognition are domain specific and require a domain specific architecture. |
| Approach: | They propose a retrieval augmented large language model for Named Entity Recognition . the model uses word-embedding over sentence-level embedding to fine tune . |
| Outcome: | The proposed model outperforms existing models on the CrossNER dataset. |
MAIR: A Massive Benchmark for Evaluating Instructed Retrieval (2024.emnlp-main)
Copied to clipboard
Weiwei Sun, Zhengliang Shi, Wu Long, Lingyong Yan, Xinyu Ma, Yiding Liu, Min Cao, Dawei Yin, Zhaochun Ren
| Challenge: | Existing IR benchmarks focus on a limited scope of tasks, making them insufficient for evaluating the latest IR models. |
| Approach: | They propose a multi-task instruction-tuned IR benchmark that includes 126 distinct IR tasks across 6 domains. |
| Outcome: | The proposed model performs better on instruction-tuned models than non-instruction-tunned models on MAIR. |
ARL2: Aligning Retrievers with Black-box Large Language Models via Self-guided Adaptive Relevance Labeling (2024.acl-long)
Copied to clipboard
| Challenge: | Existing retrievers are misaligned with large language models due to separate training processes and inherent black-box nature of LLMs. |
| Approach: | They propose a retriever learning technique that harnesses LLMs as labelers to annotate and score adaptive relevance evidence. |
| Outcome: | Extensive experiments show that ARL2 improves accuracy and reduces the cost of API calls. |
In-Context Retrieval-Augmented Language Models (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing RALM methods focus on modifying the LM architecture to facilitate incorporation of external information, complicating deployment. |
| Approach: | They propose to condition a language model on relevant documents from a grounding corpus during generation by conditioning on external knowledge sources. |
| Outcome: | The proposed method significantly improves language modeling performance and provides natural source attribution mechanism. |
Self-Knowledge Guided Retrieval Augmentation for Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown superior performance without task-specific fine-tuning due to the computational costs. |
| Approach: | They propose a method which lets LLMs refer to the questions they have previously encountered and adaptively call for external resources when dealing with new questions. |
| Outcome: | The proposed method outperforms chain-of-thought based and fully retrieval-based methods on multiple datasets and outperformed chain- of-though, chatGPT and InstructGPT. |
Generation-Augmented Retrieval: Rethinking the Role of Large Language Models in Zero-Shot Relation Extraction (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in Relation Extraction (RE) emphasize Zero-Shot methodologies, aiming to recognize unseen relations between entities with no annotated data. |
| Approach: | They propose a plug-in retrieval adjuster that allows rapid fine-tuning without accessing LLMs’ parameters. |
| Outcome: | The proposed model demonstrates comparable performance on multiple benchmarks. |