Feature
RAG evaluation dataset generator
FAQai builds an evaluation dataset from a document you supply: a question, the source context, and an expected answer. Generating that file is not the same as measuring a live chatbot.
What the evaluation export contains
The evaluation export maps each item to question, ground_truth, and contexts. ground_truth is the expected answer when the item has one, otherwise the canonical answer. Contexts come from the item context or the source chunk. The file is JSON. It does not include a score for your production system.
What it does not do
FAQai does not call your production retriever, compute faithfulness or context precision, or certify that a deployed assistant is accurate. You run those checks in your own stack. The RAG Playground answers from the processed document so you can inspect sources. That is a review tool, not a production accuracy proof.
Who can generate it
Evaluation and adversarial datasets are on Basic, Starter, and Pro. The Free plan does not include them. See pricing and the Ragas field mapping.
Illustrative record
Synthetic example, labelled as such. Not a measured benchmark.
{
"question": "How long is the return window?",
"ground_truth": "30 days from delivery.",
"contexts": ["Items can be returned within 30 days of delivery."]
}