Independent knowledge base software guide Evidence-led Updated 2026
60 synthetic cases · Excel and CSV · No signup
AI Answer Quality Test Dataset Template
Test whether a knowledge-base assistant answers from approved evidence, asks for missing context, abstains when evidence is absent, exposes conflicts, and refuses unauthorized requests.
Independent and vendor-neutral: no vendor names, affiliate ranking, prefilled scores, or email gate are built into the file.
Start HereInputsSummaryLists
IDAreaEvidence / taskStatus
01SearchRun the fixed query and preserve proofNot tested
02AccessVerify restricted content with two personasEvidence
03ExportReconcile reusable content and metadataReview
04DecisionKeep blockers separate from averagesHuman
Illustrative preview of the file structure. The download contains the full editable template.
Excel .xlsx + UTF-8 CSVeditable download
Version 1.0clear change control
August 10, 2026last updated
No signupdirect download
Inside the download
What the 60-case AI dataset tests
The value is in the method, evidence fields, visible limits, and decision controls—not in decorative blank cells.
12 each for ANSWER, CLARIFY, ABSTAIN, CONFLICT, and REFUSE using a fictional Atlas organization.
Question, source text, expected behavior, expected answer, required facts, forbidden claims, source IDs, citation, and risk.
System and version, retrieval config, prompt, seed, sources, raw answer, behavior, dimension results, critical failure, reviewer, and notes.
Behavior and every applicable quality dimension must pass, with no critical failure.
Runs, strict passes, rates, and critical failures for each category.
Correctness, groundedness, citation, authorization, schema, and safe use of N/A.
How to use it
Four steps from blank template to evidence
Keep question, approved context, expected behavior, and rubric stable for comparison.
Capture model, version, retrieval configuration, prompt, seed or temperature, and date.
Do not edit failed answers; record sources, citations, and reviewer results.
Investigate authorization, leakage, unsafe conflict, or invented policy before release.
What this template does not prove
This is an English-only, synthetic, single-turn starter pack—not an industry benchmark, production dataset, statistical estimate of rare failures, or proof of system security. Replace it with your own approved corpus for a real decision.
Use it with evidence
Related guides and companion templates
File-specific questions
Frequently asked questions
Is this a benchmark of real vendors?
No. It contains no vendor result. It is a reusable starter dataset for testing a named system under a documented configuration.
Why include conflict and refusal cases?
A useful knowledge assistant must do more than produce a plausible answer. It should expose unresolved evidence and preserve authorization boundaries.
How many runs should each case have?
Three runs are an exploratory starting point, not a statistical estimate of rare failures. Use more runs for non-deterministic systems and critical authorization or leakage cases.
Can we publish a pass rate?
Only with the dataset version, system version, configuration, sample size, raw-output policy, rubric, date, and limitations. Never present it as universal product quality.
Download, adapt, and preserve the evidence
We keep each download URL stable across version updates so your bookmarks and citations continue to work. No email address or account is required.
Published by Knowledge-Base.software · Template v1.0 · Updated August 10, 2026 · Free to adapt for internal evaluation; no resale · Editorial standards · Corrections