In the years before AI tools had a name, before anyone said “agentic workflow” in a business meeting, the pattern that determined whether an automation project delivered or stalled was almost always the same: when did the team commit to the architecture, and did that decision drive everything else?
When I worked with a class-action settlement administration company, I was asked to rebuild a returns processing and fulfillment workflow from scratch. The organization processed mail returns for class-action cases — a high-volume, rule-intensive operation where errors created legal exposure and delays created operational cost. We integrated closely with the United States Postal Service and their API interfaces to build an orchestration of mail return processing rules and triggers across multiple stages.
The most important decision in that project was not which technology to use. It was when to lock the architecture. The organization pushed for early development — stakeholders wanted visible progress, and there were natural pressures to start building before the full design was settled. The argument for waiting was simple: an automated workflow has an outsized impact on everything downstream. Every component built before the architecture is set costs more to change than if it had been built in the right place from the beginning. We built the architecture first. The system delivered with near-100% accuracy. The time saved by that upfront architectural discipline paid for the entire engagement.
ishikawa
Why AI automation projects stall
Process
No baseline workflow map
Scope grows after kickoff
Architecture
Tool chosen before design
No eval loop defined
Data
Source systems unmapped
No quality baseline
People
No clear owner post-launch
Business unit bypassed
The Tool-First Trap
The typical AI automation engagement starts with a vendor selection or a tool evaluation. Someone saw a demo. A peer company is using a particular platform. A consultant has a preferred stack. The tool is selected. Then the question is asked: what should we use it for?
This sequence produces suboptimal outcomes almost every time. It is the equivalent of buying a specific vehicle before deciding what you need to carry, where you need to go, and how often. The tool shapes the scope, the scope shapes the architecture, and the architecture that results is the one the tool supports — not the one the business process requires.
The correct sequence is the reverse: map the process, identify the decision points, define what correct output looks like in measurable terms, and then select the tool that fits that specification. When you do this, the architecture is visible before a line of code is written or a vendor contract is signed.
What the Architecture Decision Actually Covers
An architecture decision for an AI automation is not a technical diagram. It is a set of answers to operational questions most teams skip:
What does the process look like step by step, including the exceptions? Most processes, when fully mapped, are more complex than the initial description. The returns processing workflow I built involved not just the standard postal API calls but a set of edge-case rules about damaged mail, forwarding addresses, and timing windows that only surface when you map the full process before building. Every one of those edge cases had a business consequence if handled wrong.
What does correct mean, and how is it measured? An automation without a defined success metric has no way to determine whether it is working. This is the eval loop problem: the system runs, produces output, and nobody knows whether the output is right until something goes wrong downstream. The eval loop is the architectural decision that catches failure before it propagates. Most teams treat it as a post-launch addition. The organizations that get consistently good results build it into the design before the first sprint.
Who owns it after it is built? Most automation projects have a clear owner during development and an unclear one post-launch. This is where automations go wrong slowly rather than immediately. The system runs. Edge cases accumulate. Nobody has the authority to modify the rules or escalate failures to someone who does. Ownership is an architectural decision, not an HR decision — the system needs to be designed for the owner who will actually be operating it.
What happens when it fails? Failure modes are not edge cases to worry about later. They are part of the architecture. A system that routes failures to a human review queue is a fundamentally different design than a system that retries automatically or one that continues silently. Choosing between these is an architecture decision. Making it after launch means rebuilding.
What Architecture-First Actually Changes
When the architecture decision comes first — before tool selection, before development, before vendor contracts — the project timeline looks slower at the start and faster at the end. Teams that front-load the architecture decision spend more time in requirements and design. They spend less time reworking components built in the wrong direction.
More importantly, they have a document — a spec, an architecture diagram, a workflow map with decision points and failure modes — that survives beyond the initial build. When the project transitions to the team that operates it, that team has a clear picture of what the system is supposed to do and what correct looks like. That clarity is not a byproduct of good process. It is the product. The automation is the implementation.
Where This Goes Wrong in AI Specifically
AI automation projects have an additional failure mode that traditional automation projects did not: the model behavior is probabilistic, not deterministic. A rule-based automation either runs the rule or it does not. An AI automation produces outputs that vary based on the prompt, the model version, the context, and factors that may not be immediately visible. An architecture that treats AI output as reliable without an eval layer is not an architecture — it is an assumption.
The organizations that consistently get good results from AI automation are not the ones with the most sophisticated tools. They are the ones that map the process before they buy the tool, define what correct means before they write the first prompt, and design the eval loop before the first agent is deployed.
Every AI automation project that skips those decisions will face them eventually. The only question is whether they face them before deployment or after. Before is cheaper. After is educational.
