RAG vs Live Tools: Knowledge Is Not State

RAG and tools solve different problems. Retrieval-augmented generation is excellent when the model needs relevant pieces of unstructured knowledge: policies, manuals, product documentation, design docs, meeting notes, or internal wiki content. Live tools are better when the answer depends on structured, current, authoritative state.

That leads to a useful default: RAG for knowledge, tools for transactions and live state. It is not a universal law, but it is a strong starting point.

1Unstructured / relatively static2Confluence + Drive + manuals3        ↓4chunk → index → retrieve → rerank → model5 6Structured / dynamic / authoritative7Salesforce + ticketing + order state8        ↓9live tool / API call → model

The same source can support both patterns

Google Drive is a good example. “Open the Q3 pricing deck” is a direct lookup. “What has the company said over the last year about pricing strategy?” is a retrieval problem across many files. A mature system may expose both an indexed retrieval layer and a direct file-access tool against the same source.

ACL-aware retrieval is non-negotiable

Enterprise retrieval must preserve source permissions. If the user cannot access a document normally, the agent should not retrieve it into model context. ACL filtering must happen before the content reaches the model and should be driven by trusted runtime identity, not a user ID selected by the model.

1authenticated employee2      ↓3identity + groups4      ↓5retrieval query6      ↓7ACL filter8      ↓9authorized chunks only10      ↓11model context

Grounding is not authority

A retrieved refund policy can help the agent explain why a refund is or is not allowed. It should not be the enforcement mechanism. The actual decision belongs to a deterministic policy service. Documents can be stale, contradictory, or malicious; a transactional service can enforce the current rule.

PFPLabs takeaways

• RAG is for relevant knowledge; tools are for authoritative state.
• The same data source may need direct lookup and indexed retrieval.
• Retrieval must be ACL-aware before content enters model context.
• Grounding improves answers; deterministic policy enforces rules. Do not confuse evidence with authority.