
Knowledge Base Metadata for AI Search: Fields, Examples, and Validation
Knowledge base metadata for AI search gives retrieval software explicit facts about content: which product and version it covers, where it came from, whether it is published, and which access policy applies. Those facts help a system select appropriate evidence before a language model uses it to answer a question.
Consider two help articles with nearly identical wording but different export procedures. Semantic similarity may make both look relevant. The reader still needs the procedure for their product version, from an approved source they are authorized to access. Metadata provides the fields a retrieval pipeline can use to make those distinctions.
This guide focuses on internal knowledge search and retrieval-augmented generation (RAG), which supplies retrieved material to a model when generating an answer. Public website markup serves a different purpose. Google’s guidance for AI features says no special Schema.org markup is required for AI Overviews or AI Mode. Adding public structured data does not configure the fields or permissions in your own retrieval system.
Technical information last verified: November 20, 2026.
Start with the decisions retrieval must make
Take a small set of real questions from search logs, support cases, or documentation reviews. For each question, identify what would make an otherwise relevant passage unsuitable. That gives you a reason to create a field and a way to test whether it works.
- Applicability: Does the answer depend on a product, supported version, language, environment, or region?
- Publication: Should a draft or retired revision participate in this search?
- Authorization: Which source permissions must the requesting identity satisfy?
- Traceability: Can the answer identify the document, revision, and section supporting its instructions?
For an export question, product version may be essential. For a company-wide glossary, it may be irrelevant. An intended audience such as “administrators” can help route content, but it does not establish that the person asking is an administrator.
Microsoft’s RAG enrichment guidance describes adding metadata to support searches beyond semantic similarity. Use that principle selectively: keep a field when a retrieval, citation, or maintenance process actually uses it. A large inventory of unmaintained tags creates more work without establishing better answers.
Define a small metadata specification your team can maintain
A useful specification records the field’s meaning, data type, authoritative source, document or chunk level, intended use, and behavior when the value is missing. It also identifies who fixes an incorrect value. These decisions belong beside the field definition, before implementation.
Choose core fields and conditional extensions
The following is a starting design menu, not a universal platform schema. Establish identity and source traceability first. Add applicability, governance, and security fields according to the collection’s actual requirements. Field names below are illustrative.
| Document fields and type | Source and responsible team | Purpose and missing-value rule |
|---|---|---|
document_id, revision: strings | Source connector or publishing system; ingestion owner | Identify the source and indexed revision. Hold records whose identity cannot be established. If the source lacks revision IDs, define an ingestion fingerprint or equivalent tracking method. |
title, source_url: strings | Published source; documentation owner | Label and locate supporting evidence. Repair missing citation data rather than generate a plausible URL. |
status: controlled value | Publishing workflow; content owner | Distinguish states such as draft, published, and retired. Unknown status must not silently become published. |
product: string; applies_to_versions: array of strings | Approved documentation and release catalog; product documentation owner | Constrain version-dependent guidance. Hold records missing required scope; absence must not mean all versions. |
language: controlled string | Validated source locale; localization owner | Support language selection. Route unknown or conflicting values for correction; define any language fallback explicitly. |
owner_id: string; reviewed_at: timestamp | Ownership directory and review workflow; content operations | Route maintenance and document review. Keep an unknown review date unknown; do not replace it with ingestion time. |
tenant_id: string; authorized principal or group references: identifier list | Trusted source permissions and identity systems; security/platform owner | Support the authorization mechanism where needed. Missing required permission bindings keep protected content out of normal retrieval. |
Then define the chunk fields: a record identifier, a link to the parent document and revision, and a section or page locator when available. Their source is the ingestion pipeline. A missing optional section label can fall back to the document title; a missing parent relationship needs repair before the passage is used as traceable evidence.
Other fields are conditional. An effective date matters when a policy changes on a particular day. A region matters when instructions differ by location. A feature identifier helps distinguish similarly named functions. Avoid adding them to every record merely because another organization uses them.
Define values precisely enough to validate
Choose one representation for each concept. Define status values, locale conventions, timestamp formats, and whether a field permits one value or several. Treat product versions as identifiers with documented comparison rules: a version such as 3.10 must not become the floating-point number 3.1.
A retrieval metadata schema describes your field model. JSON Schema can validate object properties and required fields; it is different from Schema.org markup for web pages. Its documentation also distinguishes an absent property from a property containing null. Your ingestion rules must define how both affect eligibility.
Validate meaning as well as syntax. A well-formed version string can still name the wrong release. A valid timestamp cannot prove a review occurred. A supplied group identifier cannot prove that the source was authorized to grant access to that group.
Map the logical model to the chosen platform before rollout. For example, Cloudflare’s AI Search metadata documentation specifies its own supported types, reserved names, limits, and re-indexing behavior. Check equivalent rules in your service rather than assuming a nested JSON example can be submitted unchanged.
Preserve source context when documents become chunks
A chunk is a passage available for retrieval. It may be returned without the surrounding article, so it needs enough context to remain identifiable and applicable. Preserve the relationship to its source document and revision, along with the relevant product scope, publication state, and trusted permission binding.
Microsoft’s index-projection documentation shows how parent fields can be repeated on child chunks or handled through other index arrangements. It explicitly calls for permission fields to reach protected chunks when those fields support query-time filtering. Inspect the records actually written to the index; the presence of metadata in the original document does not prove that the mapping succeeded.

Set inheritance and override rules
Use a simple default: document identity, source revision, and document-wide scope remain associated with every derived chunk. The section heading and locator vary by passage. A subsection may narrow applicability when the source explicitly supports that difference and the pipeline validates it.
For example, a section covering one operating system can carry that narrower scope. It should not acquire a different tenant or broader access rights because extracted text contains an apparent permission field. Resolve contradictions at the source or ingestion boundary rather than letting the answer model choose which metadata to trust.
Keep warnings with the steps they qualify and table headers with their relevant values. A version label cannot repair a passage that omits a necessary prerequisite. Test chunk boundaries against the content type instead of imposing one token length across procedures, reference tables, and policies.
Plan for changed chunk identifiers after reprocessing. Keep source identity, revision, and location separate from the current chunk key so that updates can replace obsolete passages and citations can remain traceable.
Connect each field to its job in retrieval
A field can be stored successfully and still have no effect on a query. Document how each selected field is used at indexing time, query time, and answer generation.
| Job | Example | What to verify |
|---|---|---|
| Eligibility filtering | Published instructions applicable to the selected version | The field is indexed for the required filter operation and the query applies it. |
| Keyword matching | A product code, feature name, or approved alias | The relevant text field and query configuration support the match. |
| Embedding context | A useful title or product description added to passage text | The embedding input includes the intended context and evaluation supports the choice. |
| Ranking or reranking | Preference for a relevant authoritative source within the eligible set | The ranking mechanism uses the signal without admitting ineligible material. |
| Answer context and citations | Title, applicable version, source revision, and locator | The selected information reaches the answer layer and supports the displayed citation. |
The Amazon Bedrock S3 connector documentation illustrates this distinction. Metadata can be stored for filtering without being included in embedding input. Its includeForEmbedding setting controls whether a metadata key-value pair is added to the text used for embedding. That choice can influence similarity; it does not establish that adding every field will improve results.

Evaluate keyword and semantic retrieval for your actual questions. Microsoft’s hybrid search documentation describes combining full-text and vector results, including scenarios where exact product codes and specialized terms benefit from keyword matching. Metadata constraints still need their own configuration within the chosen retrieval approach.
Give tags, synonyms, and entities distinct responsibilities
Use tags for controlled categorization, such as reporting or exports. Use an entity identifier to refer consistently to a specific product or feature even when its display name changes. Use approved synonyms or aliases to connect the language readers use with the language in the documentation.
Keep aliases scoped to their meaning. “Download an activity report” may be an approved alternative to “export an activity report”; “delete an activity report” expresses a different task. Test both the match you want and the nearby match you must avoid.
Microsoft’s synonym-map guidance describes expanding queries through mappings on string fields. Treat that as a specific search capability. An alias stored in an unused metadata field does not automatically expand queries or alter embeddings.
Keep authorization separate from inferred relevance
Get the reader’s security context from the authenticated application and the content’s permissions from a trusted source or policy mechanism. An AI system may propose a topic filter, but it must not create tenant membership or grant access by generating a group name.
Microsoft’s document-level access guidance separates query-time checks from the trustworthiness of indexed identities and permission values. AWS also warns in its S3 connector documentation that synchronized content can be available to callers with the relevant retrieval permission, including content restricted in the source. Verify the connector’s enforcement path rather than assuming source access rules transfer automatically.
Apply authorization before protected material reaches an unauthorized answer-generation context or user. Include titles, snippets, citations, and downstream copies in the design. If a required security binding is missing or cannot be trusted, keep that material unavailable through the normal retrieval path.
Worked example: retrieve the correct export instructions
Consider a fictional product called CedarDesk. Versions 3.2 and 2.8 are both supported, but their activity-report export procedures differ. The knowledge base contains these three source records:
| Source record | Applicable product version | Publication status |
|---|---|---|
kb-export-32, revision 4 | 3.2 | Published |
kb-export-32, revision 3 | 3.2 | Retired |
kb-export-28, revision 2 | 2.8 | Published |
All three records belong to tenant demo-a and require membership in its reporting group. A trusted connector and policy system establish those restrictions. They are independent of the article’s descriptive tags.
Represent the document and its derived passages
This JSON is an illustrative logical model, not a payload accepted by every search API. It shows metadata relationships; passage text is omitted for clarity. The nested chunks do not prescribe a physical index layout, and the example.com URL is a fictional revision-specific source locator.
{
"document_id": "kb-export-32",
"revision": "4",
"title": "Export an activity report",
"source_url": "https://example.com/help/activity-export/3.2/rev-4",
"product": "CedarDesk",
"applies_to_versions": ["3.2"],
"status": "published",
"language": "en-US",
"chunks": [
{
"chunk_id": "kb-export-32:r4:c1",
"section": "Before you export"
},
{
"chunk_id": "kb-export-32:r4:c2",
"section": "Create the export"
}
]
}
The publishing system supplies the identity, revision, title, locator, and status. The documentation owner verifies version applicability against the release catalog. Each indexed chunk retains the parent document’s identity, revision, source locator, product, applicable versions, status, and language through the platform’s field mapping or an equivalent relationship.
Authorization remains a separate trusted association between the actual source object, tenant, and permitted group. The ingestion pipeline must preserve that association for both chunks. Supplying kb-export-32 as a document ID cannot itself prove that a record is entitled to inherit the source’s permissions.
Validate the relationship before publishing the records. If an extractor labels the second chunk as applying to 2.8 while its source applies only to 3.2, hold the affected records for correction. Do not combine the conflicting values into broader applicability. A missing optional section label can use the document title, but it must not produce an invented section anchor.
Follow one question through retrieval
An authenticated member of the demo-a reporting group asks, “How do I export an activity report in CedarDesk 3.2?” The application resolves identity and membership from its trusted security context. It takes the product version from the question or a reliable application setting, keeping that relevance context separate from access grants.
For this example, eligibility requires authorized access, published status, product CedarDesk, and an explicit 3.2 applicability value. The chosen English-language collection also uses en-US. Among the three fixtures, only revision 4 of kb-export-32 qualifies. Retrieval then selects the relevant passages within that eligible material.
The answer should preserve any prerequisites from “Before you export” needed to follow “Create the export,” then cite the selected source revision. In a production system, use the actual source locator and supported revision mechanism; never manufacture a versioned URL because the example has one.
A reader asking about 2.8 should receive its published revision 2, even though a newer product version exists. If the version is unknown and the instructions depend on it, request clarification. This example uses exact version identifiers and makes no assumption that 3.2 instructions also apply to 3.1.
Test applicability, authorization, and answer support
Create a small evaluation set from representative questions and known source records. Record the requesting identity, relevant application context, eligible sources, required supporting passages, and prohibited disclosures. That gives reviewers something more concrete than whether an answer sounds helpful.
The following are proposed tests for the fictional CedarDesk collection. They describe expected outcomes, not measured results from an executed search system.
| Test | Input or change | Expected outcome |
|---|---|---|
| Correct published revision | Authorized reporting user asks about 3.2. | Use relevant chunks from kb-export-32, revision 4. Exclude retired revision 3 and the 2.8 article; cite the selected revision. |
| Older supported version | The same user asks about 2.8. | Use kb-export-28, revision 2. A newer timestamp or product version does not override applicability. |
| Missing or unmatched version | No version is established, or the request specifies 3.1. | Clarify missing context or explain that the collection lacks applicable guidance. Do not assume compatibility. |
| Alias boundary | Compare “download an activity report” with “delete an activity report.” | The approved download alias can retrieve export guidance. The delete request must not be answered with export instructions. |
| Missing required scope | Remove applies_to_versions from a staged record. | Validation holds the record and derived chunks until repaired. Missing does not mean applicable to every version. |
| Insufficient permission | A reader in demo-a lacks the reporting group. | No protected passage, title, snippet, or citation is exposed. An empty response is not evidence of successful task completion. |
| Forged security context | A demo-b reader or model-generated filter supplies demo-a and its reporting group. | Trusted authorization context prevails. The supplied values grant no access. |
| Revoked permission | Remove the reader’s grant and exercise the documented permission-update path. | After the required update is verified, affected chunks are unavailable. Record propagation behavior and apply the defined deny or hold policy if required authorization freshness cannot be established. |
Score retrieval separately from the answer
First check whether retrieval found applicable, authorized passages and covered the information needed to answer. Then check whether the generated answer is correct, stays supported by those passages, and cites the evidence for its actual claims. Returning the right document does not prove that the model used it correctly.
Amazon Bedrock’s RAG evaluation documentation separates retrieve-only metrics, including context relevance and coverage, from answer metrics such as correctness, faithfulness, and citation quality. Bedrock’s context-coverage metric requires ground-truth reference text in the evaluation dataset. Choose measures that match your task, and check access exclusions directly instead of treating a high answer-quality score as a security test.

For a before-and-after comparison, keep the query set, source corpus, identities, and other retrieval settings controlled. Change the metadata design you want to evaluate, review both gains and regressions, and record which questions still fail. Set acceptance criteria from the consequences of those failures; there is no universal percentage that proves a knowledge base is ready.
Investigate empty results without widening access
When a query should have an answer but returns none, inspect the source record, inherited chunk fields, data types, allowed values, and applied filters. Confirm that the expected record reached the index and is actually eligible. A misspelled version or unmapped field can resemble a missing article.
Also check the engine’s filter execution behavior. Microsoft’s vector-filter documentation explains how prefiltering and postfiltering differ, including cases where restrictive filters and small candidate sets can reduce returned results. Test the documented options for your workload without assuming one setting is best everywhere.

A documented fallback might relax an optional topic preference or ask the reader for missing context. It must not remove tenant restrictions, waive required permissions, or silently substitute instructions for an incompatible version. Record the failed query so the team can distinguish a content gap from a configuration problem.
Keep metadata aligned with source changes
Give the content owner responsibility for meaning and applicability, and the platform owner responsibility for propagation and enforcement. A corrected field in the CMS is only the beginning: the serving index and any dependent copies must reflect the change.
Track the event each timestamp represents
Keep source modification, substantive review, policy effectiveness, and index ingestion separate. A pipeline can re-index a three-year-old policy today without anyone reviewing it. Conversely, an older article can remain correct for a supported product version. Use timestamps with status and applicability instead of treating the newest record as automatically authoritative.
Define which events require review: a product release, changed procedure, revised access policy, or evidence that answers are failing. Assign the review to an owner and make unresolved work visible. Choose review intervals according to content risk and change frequency, rather than applying the same calendar to every collection.
Verify updates through the serving system
For a changed article, confirm the correct revision and derived chunks become available and superseded passages leave the normal retrieval path. For deletion or retirement, verify that every affected chunk is removed or made ineligible. Check the source mapping and deletion configuration; a successful upload of the replacement alone does not establish that older records disappeared.
Distinguish updates to stored metadata from changes to embedding input. Amazon Bedrock’s synchronization documentation describes source synchronization and a metadata-only optimization that can reuse vectors under specified conditions. Determine whether your change affects only stored fields or also the text used to create embeddings, then verify that the serving index reflects the intended change. Do not assume all metadata edits require the same processing.

Test permission changes separately from content changes. Microsoft’s access-control documentation describes different permission mechanisms and cautions against assuming immediate revocation or a universal freshness guarantee. Verify your connector’s propagation and caching behavior. Define how cached results, saved answers, and conversation history are protected when access changes; updating retrieval does not automatically rewrite existing copies.
Change the field model deliberately
Track a schema revision when field meanings, types, or required values change. Update validation and query logic together, backfill existing records where needed, and rerun representative queries before switching the serving collection. Otherwise, new and old records may interpret the same filter differently.
AI-assisted enrichment can propose topics, entity candidates, or short descriptions. Validate those proposals against the allowed vocabulary and source content. Keep authorizations, tenant identity, publication approval, ownership assignments, and completed-review claims tied to authoritative systems or explicit approval workflows.
Start with one article family and a set of questions that currently fail. Define the smallest useful field specification, assign its sources and owners, and trace representative records from publishing through chunking to the answer. Expand after the team can explain both why the correct source was selected and why an inapplicable or unauthorized source was excluded.



