2024-09-12
arXiv

Enhancing Q&A Text Retrieval with Ranking Models: Benchmarking, fine-tuning and deploying Rerankers for RAG

Gabriel de Souza P. Moreira , Ronay Ak , Benedikt Schifferer , Mengyao Xu , Radek Osmulski
This paper benchmarks and evaluates various ranking models for enhancing the accuracy of text retrieval in question-answering tasks, introducing a new model, NV-RerankQA-Mistral-4B-v3, that significantly improves accuracy. It also discusses the trade-offs between model size, accuracy, and system requirements in real-world applications.
Ranking models play a crucial role in enhancing overall accuracy of text retrieval systems. These multi-stage systems typically utilize either dense embedding models or sparse lexical indices to retrieve relevant passages based on a given query, followed by ranking models that refine the ordering of the candidate passages by its relevance to the query. This paper benchmarks various publicly available ranking models and examines their impact on ranking accuracy. We focus on text retrieval for question-answering tasks, a common use case for Retrieval-Augmented Generation systems. Our evaluation benchmarks include models some of which are commercially viable for industrial applications. We introduce a state-of-the-art ranking model, NV-RerankQA-Mistral-4B-v3, which achieves a significant accuracy increase of ~14% compared to pipelines with other rerankers. We also provide an ablation study comparing the fine-tuning of ranking models with different sizes, losses and self-attention mechanisms. Finally, we discuss challenges of text retrieval pipelines with ranking models in real-world industry applications, in particular the trade-offs among model size, ranking accuracy and system requirements like indexing and serving latency / throughput.
2024-07-01
arXiv

Searching for Best Practices in Retrieval-Augmented Generation

Zhenghua Wang , Xuanjing Huang , Xiaoqing Zheng , Changze Lv , Ruicheng Yin
The paper investigates different RAG approaches to identify optimal practices that balance performance and efficiency, and shows that multimodal retrieval techniques can enhance question-answering and content generation.
Retrieval-augmented generation (RAG) techniques have proven to be effective in integrating up-to-date information, mitigating hallucinations, and enhancing response quality, particularly in specialized domains. While many RAG approaches have been proposed to enhance large language models through query-dependent retrievals, these approaches still suffer from their complex implementation and prolonged response times. Typically, a RAG workflow involves multiple processing steps, each of which can be executed in various ways. Here, we investigate existing RAG approaches and their potential combinations to identify optimal RAG practices. Through extensive experiments, we suggest several strategies for deploying RAG that balance both performance and efficiency. Moreover, we demonstrate that multimodal retrieval techniques can significantly enhance question-answering capabilities about visual inputs and accelerate the generation of multimodal content using a "retrieval as generation" strategy.