Back to blog

Building Production-Grade AI Agents · Part 6 of 15

Deterministic vs Probabilistic Orchestration

The most important architecture decision in an agent system is deciding which parts should not be agentic at all.

Chris Eberl

Chris Eberl

Founder • Engineering Leader, GenAI, Data, ML

Production AI AgentsSep 14, 20265 min read
Deterministic vs Probabilistic Orchestration

Agents are powerful because they can choose a path based on ambiguous observations. Traditional workflow engines are powerful because they execute known paths reliably. Production systems work best when we stop treating those as competing paradigms and use each where it is strongest.

A simple heuristic: if the graph is known and the data varies, prefer a deterministic workflow. If the graph itself may change based on semantic observations, an agent or planner becomes useful.

Sequential, parallel, and planned work

If step B requires the result of step A, execute sequentially. If two calls are known to be independent, parallelize them in code rather than paying model latency for serial execution. If the model must discover which subtasks are required, let it produce a structured plan and let deterministic software schedule that plan.

System flow
1User: “Update Acme and tell the sales team.”2        ↓3semantic planner4        ↓5[6  update_salesforce,7  notify_slack(depends_on=update_salesforce)8]9        ↓10deterministic orchestrator

The handoff to durable workflow

Once an agent has decided on a known business operation, the remaining execution often belongs in a workflow engine. A financial action might require payment update, ledger write, order-state update, and notification. The agent does not need to individually reason through a known transaction every time. It can hand that operation to a durable workflow with explicit retries and state.

Router versus planner

A router chooses a domain or capability set. A planner decides which steps are needed. That distinction matters when requests span multiple domains. “Update the opportunity and let the team know” should not be forced through a single Salesforce-or-Slack router; it is a multi-intent plan.

Deterministic routing after probabilistic classification

Sometimes intent cannot be known from UI state or business rules. In that case, an LLM can classify into a small structured set of routes. The classification is probabilistic, but once it returns {domain: salesforce}, normal code can deterministically expose only the Salesforce tool set.

PFPLabs takeaways

  • Do not make known workflows agentic for style points.
  • Parallelize independent work in code. Model turns are expensive coordination primitives.
  • Separate routing from planning. One chooses a domain; the other constructs a task graph.
  • Hand known durable processes to workflow engines. Agents decide; workflows execute.

Further reading

AWS Step Functions: Choosing workflow types

Read next

Newsletter

New posts, straight from Chris

A short note from me whenever a new article goes live — product engineering, AI workflows, IoT, indie apps, and engineering leadership. No spam, unsubscribe anytime.

By subscribing, you agree to our Privacy Policy. We do not share your email.