Dataset on RAG Pipeline Evaluation for Retrieval and Generative Response Accuracy Testing

Citation Author(s):
Pruthvi Raj
Venkatesh
NIT Warangal
Radha Krishna
P
NIT Warangal
Submitted by:
Pruthvi Raj Ven...
Last updated:
Sat, 08/31/2024 - 13:01
DOI:
10.21227/hg2s-f350
Data Format:
*.json;
License:
Creative Commons Attribution
423 Views
Categories:
Artificial Intelligence
Keywords:
RAG, RRR, Generative AI, RAGAS, Retreival Accrecy, LLM
0
0 ratings - Please login to submit your rating.
ACCESS DATASET CITE SHARE/EMBED

Abstract 

This dataset has been meticulously curated to evaluate the efficiency of Retrieval-Augmented Generation (RAG) pipelines in both retrieval and generative accuracy, with a particular focus on scenarios involving overlapping contexts. The dataset comprises two primary components: Motor data and Employee data. The Motor dataset includes master data of various motor models along with their corresponding manuals, linked by the motor's model name. Similarly, the Employee dataset encompasses employee master data and associated policy documents, linked by department. By providing a diverse and contextually rich set of information, this dataset serves as a comprehensive resource for testing RAG pipelines' capabilities in handling complex queries and generating precise responses in domains where context overlap is prevalent.

Instructions: 

Overview

This dataset is designed to facilitate testing and evaluation of Retrieval-Augmented Generation (RAG) pipelines. It is divided into two main sections: Employee and Motor.

Dataset Files

  1. IeeeEmployeeData.json: Contains employee information.
  2. IeeeEmployeePolicyData.json: Contains departmental policies.
  3. IeeeMotorData.json: Contains motor information.
  4. IeeeMotorManualData.json: Contains motor manuals.

How to Use

  1. Load the JSON Files: Parse the JSON files to access the data.
  2. Link Data: Use the DepartmentName in IeeeEmployeeData.json to link with Department in IeeeEmployeePolicyData.json, and ModelName in IeeeMotorData.json to link with Model in IeeeMotorManualData.json.
  3. Test RAG Pipelines: Use the overlapping context in the manuals and policies to test and evaluate RAG models for accurate context retrieval and question answering.

Contact

For any questions or further information, please contact:

Dataset Files

LOGIN TO ACCESS DATASET FILES

Documentation

AttachmentSize
File Dataset Instruction 5.99 KB

 

QUESTIONS?