Software Architecture vs AI Agent Architecture: The Same Patterns with Different Actors

Photo by richard_clyborne on Flickr, CC BY 2.0

Many software engineers look at AI right now and see a wave they have to catch. There is a new framework every month, a new vocabulary, and a new set of job titles. It is easy to assume that the experience already earned is about to be replaced.

I do not think that is what is happening. Most of what a software engineer already knows still applies to AI systems. Architecture is the clearest example, so that is where this post starts.

Architecture answers one question: who decides what happens next.

There are only two answers, and both already have names. The first is orchestration. One component holds the plan and tells the other components what to do. The second is choreography. No component holds the plan. Each one watches for something to happen and reacts to it.

Microservices use both patterns. AI agent systems use both patterns too. Nothing about the architecture is new. The diagrams are the same ones the industry has been drawing for years.

What changed is the actor inside the boxes.

A service is deterministic. Give it the same input and it produces the same output every time, and it fails the same way every time. An agent is probabilistic. It samples its answer from a model, so the same input can take a different path today than it took yesterday.

The pattern decides who is in control. The actor decides how predictable the result is. Almost every practical difference between the two worlds comes from the actor.

Software Architecture: Orchestration and Choreography

In orchestration, one service owns the sequence. It calls payment, then stock, then shipping. The whole flow lives in one place and can be read from top to bottom. Changing the flow means changing that one service.

In choreography, no service owns the sequence. Each service publishes an event when something happens, and any service that cares subscribes to it. Checkout publishes an order event, and payment and shipping react on their own. Adding a fourth service costs nothing, because nobody has to be told it exists.

Orchestration and choreography in software architecture

Each pattern has a price. Orchestration is easy to read and hard to extend, because everything depends on the component in the middle. Choreography is easy to extend and hard to read, because the flow is not written down anywhere. To follow it, you have to read every service and check which events it publishes and which it subscribes to.

That shows up in daily work. An orchestrated flow is the one a new engineer can follow on their first day. A choreographed flow is the one a team extends in an afternoon and then spends a week explaining when something goes missing. A simple rule of thumb helps here. Orchestrate what a person would describe as a process. Choreograph what they would describe as a consequence.

AI Agent Architecture: The Same Two Patterns

In an orchestrated agent system, one component reads the incoming message and decides which agent should handle it. That component is usually called a router or a supervisor, and it also decides how much of the conversation each agent may see.

In a choreographed agent system, there is no router. The agents share a workspace, often called a scratchpad or shared state. Each agent reads what is there, adds what it can, and stops. The plan is never written down. It emerges from the order in which the agents happen to act.

The same two patterns in AI agent architecture

In production, the orchestrated version is the common one. An intent router sits in front of a registry of domain agents, and each agent has its own tools and its own slice of the conversation. The design is deliberately unexciting, because the cost of a wrong route is high. If the router picks the wrong agent, the person asking does not get the answer they came for.

The choreographed version looks impressive in a demo. It is also the version that can run fourteen turns and produce nothing, because no component was responsible for the result. Underneath either pattern the agents still have to talk to each other in a format both sides understand. That is a separate problem with its own long history.

The Actor Is the Difference

The table below puts the two worlds side by side. The rows about who is in control look almost the same. The rows about how the actor behaves do not.

Software and AI agent architecture compared, starting from the actor

A deterministic actor fails loudly. A service that routes to the wrong place returns an error, and that error appears in a trace. The trace is enough to explain what happened, because the code that ran is the code you can read in the repository. Retrying is safe if the call is idempotent, and the retry either works or fails the same way again.

A probabilistic actor fails quietly. An agent that routes to the wrong place still produces a fluent, confident answer. Nothing turns red. The failure is only visible to a person who reads the answer and knows enough to doubt it. A trace helps less here, because it records what the agent said and not why it said it. Retrying does not repeat the same run. The model samples again, so the second answer may be right, or wrong in a new way. Neither outcome means the problem is fixed.

This is why choreography is riskier with agents than with services. In both worlds it removes the single place where the plan lives. With services that mainly costs readability. With agents it also removes the only component that could have checked the answer before the user saw it.

Making a Probabilistic Actor More Predictable

The actor cannot be made deterministic. The system around it can. Three techniques do most of that work.

Validation checks the answer before anything uses it. The output has to match a schema, carry the required fields, and stay inside the allowed values. An answer that fails validation never reaches the user.

Guardrails limit what the agent may do before it acts. They set which tools it may call, which topics it may answer, and what it has to refuse. A guardrail turns an open question into a small set of allowed moves.

A verifier checks the answer afterwards against the source it came from, using a rule or a second model. This is the component that catches the confident wrong answer described above.

None of this makes the model deterministic. It moves the checks out of the model and into code, where the behaviour is predictable again. Each technique deserves its own article, and that is what the next post is about.

Conclusion

Software architecture and AI agent architecture are the same architecture. The patterns did not change when the components started calling a model. Only the actor changed, from something deterministic to something probabilistic.

The practical advice follows from that. Start with orchestration, because one component holding the plan is the one component you can read. Move a part to choreography when you can name what reacts to it and what happens when nothing does. In software, the question is how tightly the parts depend on each other. In an agent system there is a second question: if an agent answers wrongly, will anything catch it before the user sees it?

For a software engineer moving into AI, that is good news. The architecture knowledge transfers. What has to be learned is the behaviour of the new actor, and the checks that keep it honest.