RAG for Business: AI over Trusted Company Knowledge

Define the decisions the system must support

Start with concrete questions, users and acceptable evidence. A support assistant, sales copilot and policy search tool need different sources, permissions, freshness and failure behavior.

Specify when the system should answer, ask a clarifying question, cite sources or decline because evidence is insufficient.

Build a governed source layer

Assign owners to source collections and define approval, versioning, retention and update rules. Remove duplicates, obsolete documents and contradictory copies before indexing.

• Authoritative source and document owner
• Audience and access classification
• Effective and expiry dates
• Review status and update frequency
• Traceable source URL or record identifier

Preserve permissions through retrieval

A user must never retrieve content they could not access in the source system. Apply identity and authorization filters before retrieval, protect indexes and logs, and test cross-role leakage.

Treat prompts, retrieved passages and feedback as potentially sensitive operational data.

Evaluate retrieval separately from generation

Measure whether the correct evidence was retrieved before judging the final prose. Build a representative question set with expected sources and include ambiguous, outdated and unanswerable cases.

Track retrieval recall, citation correctness, groundedness, answer usefulness and safe refusal. A fluent answer with the wrong evidence is a serious failure.

Operate knowledge freshness

Monitor failed queries, unused sources, stale content, latency and cost. Route feedback to source owners and re-run regression evaluations after embedding, chunking, model or source changes.