Rankify: A Comprehensive Python Toolkit for Retrieval, Re-Ranking, and Retrieval-Augmented Generation (2026.acl-demo)
Copied to clipboard
Abdelrahman Abdallah, Bhawna Piryani, Jamshid Mozafari, Andreas Herzinger, Jamie Holdcroft, Adam Jatowt
| Challenge: | Rankify unifies retrieval-augmented generation (RAG) and retrieval based question answering systems. |
| Approach: | They propose an open-source Python toolkit that unifies retrieval-augmented generation in a single modular framework. |
| Outcome: | The proposed framework unifies retrieval-augmented generation (RAG) tools in a single modular framework. |
Similar Papers
RAGVUE: A Diagnostic View for Explainable and Automated Evaluation of Retrieval-Augmented Generation (2026.eacl-demo)
Copied to clipboard
| Challenge: | Existing tools for evaluating RAG systems often collapse heterogeneous behaviors into single scores. |
| Approach: | They propose a diagnostic framework for automated, reference-free evaluation of RAG pipelines. |
| Outcome: | The proposed framework decomposes RAG behavior into retrieval quality,answer relevance and completeness, strictclaim-level faithfulness, and judge calibration. |
BERGEN: A Benchmarking Library for Retrieval-Augmented Generation (2024.findings-emnlp)
Copied to clipboard
David Rau, Hervé Déjean, Nadezhda Chirkova, Thibault Formal, Shuai Wang, Stéphane Clinchant, Vassilina Nikoulina
| Challenge: | Retrieval-Augmented Generation allows to enhance Large Language Models with external knowledge. |
| Approach: | They propose a library that allows to benchmark and standardize RAG experiments. |
| Outcome: | The proposed library is an end-to-end library for reproducible research standardizing RAG experiments. |
RankCoT: Refining Knowledge for Retrieval-Augmented Generation through Ranking Chain-of-Thoughts (2025.acl-long)
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) models enable Large Language Models to access external knowledge. |
| Approach: | They propose a knowledge refinement method that incorporates reranking signals to generate CoT-based summarization based on query and retrieval documents. |
| Outcome: | RankCoT generates CoT-based summarization based on query and all retrieval documents . Rank CoT incorporates a self-reflection mechanism that refines the outputs . |
Retrieval Enhancements for RAG: Insights from a Deployed Customer Support Chatbot (2026.eacl-industry)
Copied to clipboard
Daniel González Juclà, Mohit Tuteja, Marcos Esteve Casademunt, Keshav Unnikrishnan, Yasir Usmani, Arvind Roshaan
| Challenge: | a persistent gap remains between Recall@10 and Recall @50 across datasets . |
| Approach: | They evaluate embedding model comparison, Reciprocal Rank Fusion and embedded concatenation techniques to improve retrieval quality. |
| Outcome: | The proposed methods outperform traditional cross-encoders in identifying high-relevance passages. |
RAGAs: Automated Evaluation of Retrieval Augmented Generation (2024.eacl-demo)
Copied to clipboard
| Challenge: | RAGAs are a framework for reference-free evaluation of Retrieval Augmented Generation (RAG) pipelines. |
| Approach: | They propose a framework for reference-free evaluation of Retrieval Augmented Generation pipelines. |
| Outcome: | RAGAs can be used to evaluate RAG pipelines without human annotations . the framework can be useful for faster evaluation cycles given the fast adoption of LLMs based on human annotation. |
ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems (2024.naacl-long)
Copied to clipboard
| Challenge: | Evaluating retrieval-augmented generation systems relies on hand annotations for input queries, passages to retrieve, and responses to generate. |
| Approach: | They propose an automated evaluation framework for retrieval-augmented generation (RAG) ARES fine tunes lightweight LLM judges on synthetically generated queries and answers . |
| Outcome: | The proposed framework evaluates RAG systems using only human annotations . it can be used to improve system understanding and create targeted solutions . |
InfoGain-RAG: Boosting Retrieval-Augmented Generation through Document Information Gain-based Reranking and Filtering (2025.emnlp-main)
Copied to clipboard
Zihan Wang, Zihan Liang, Zhou Shao, Yufei Ma, Huangyu Dai, Ben Chen, Lingtao Mao, Chenyi Lei, Yuqing Ding, Han Li
| Challenge: | Retrieval-Augmented Generation (RAG) frameworks struggle with identifying whether retrieved documents meaningfully contribute to answer generation. |
| Approach: | They propose a document-related metric to quantify the contribution of retrieved documents to correct answer generation. |
| Outcome: | The proposed framework outperforms existing approaches on both single and multiple retrieval paradigms. |
RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation (2024.emnlp-demo)
Copied to clipboard
Xuanwang Zhang, Yun-Ze Song, Yidong Wang, Shuyun Tang, Xinfeng Li, Zhengran Zeng, Zhen Wu, Wei Ye, Wenyuan Xu, Yue Zhang, Xinyu Dai, Shikun Zhang, Qingsong Wen
| Challenge: | Existing research on Retrieval Augmented Generation (RAG) does not address the problem of hallucinations and real-time updating of knowledge. |
| Approach: | They propose a modular open-source library to equip LLMs with external knowledge. |
| Outcome: | The proposed approach reduces the need for expensive open-source tools and lacks fair comparisons between novel RAG algorithms. |
Retrieval-augmented Generation across Heterogeneous Knowledge (2022.naacl-srw)
Copied to clipboard
| Challenge: | Existing methods for retrieving knowledge from a single source homogeneous corpus have been gaining increasing attention in the field of natural language processing (NLP) however, they still suffer from the following drawbacks: (i) They are usually trained offline, making the model agnostic to the latest information, e.g., asking a chat-bot about COVID-19. |
| Approach: | They propose to use a single-source homogeneous corpus to generate retrieval-augmented generation models that can learn from the pre-training corpus. |
| Outcome: | The proposed methods have been applied to various knowledge-intensive NLP tasks, but most of the work has focused on retrieving unstructured text documents from Wikipedia. |
Rethinking Retrieval-Augmented Generation as a Cooperative Decision-Making Problem (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing RAG systems rely on ranking-centric, asymmetric dependency paradigms to generate results. |
| Approach: | They propose a framework that treats the reranker and the generator as peer decision-makers rather than being connected through an asymmetric dependency pipeline. |
| Outcome: | The proposed framework treats the reranker and the generator as peer decision-makers rather than being connected through an asymmetric dependency pipeline. |