AI Agent Development: From Workflow Mapping to Production

Start with one bounded process.

Interview the people who perform it and collect real examples. Record the inputs they receive, systems they open, information they search for, decisions they make, exceptions they encounter, actions they take, and conditions that cause escalation.

A useful workflow map separates five types of work.

This decomposition prevents a common design mistake: giving a model control over operations that ordinary software can perform exactly.

Capture the exceptions

Experienced employees spend much of their time handling cases that do not match the standard procedure.

Those cases are disproportionately useful during design.

Collect examples such as missing records, contradictory records, stale documents, ambiguous requests, tool outages, permission restrictions, duplicate actions, unusual customer histories, and cases where the correct outcome is to stop.

These examples later become part of the eval set.

An agent is justified when the correct next step depends on information discovered during execution and the path cannot be reliably encoded in advance.

Anthropic's agent engineering guidance recommends beginning with the simplest architecture that satisfies the task. Its distinction is useful: workflows follow predefined code paths, while agents dynamically direct their own process and tool use.

A new system needs something to beat.

Measure the current process before replacing it.

Useful baseline measures include:

• completion rate;
• average handling time;
• human minutes per case;
• escalation rate;
• rework rate;
• error rate;
• cost per completed case;
• customer wait time;
• backlog;
• percentage of cases with complete evidence.

The exact set depends on the workflow.

The baseline also helps expose cases where automation creates little economic value. A process performed 20 times per month with five minutes of labor per case has a different automation case from one performed 50,000 times per month.

"Build an agent that handles support" is not testable.

A success contract describes what completion means.

For a billing-dispute agent, the contract requires:

1. 1correct customer identity;
2. 2correct invoices retrieved;
3. 3current policy retrieved;
4. 4dispute classified correctly;
5. 5permitted action selected;
6. 6approval triggered above a threshold;
7. 7tool called with valid arguments;
8. 8resulting account state verified;
9. 9customer response consistent with the resulting state.

A run that produces a polished message while refunding the wrong account fails.

A production agent commonly contains these layers:

The architecture should identify the authoritative source for each fact. Customer balance belongs in the billing system. Policy belongs in the policy repository. A model-generated summary is useful context, but it should not silently replace the system of record.

Agent evals should represent the work the system will face.

Start with historical cases and create categories.

Anthropic's 2026 eval guidance notes that agents are harder to test because they operate across multiple turns, use tools, modify state, and react to intermediate results. OpenAI's current agent-evaluation guidance likewise treats traces as a first-class evaluation surface.

A read-only prototype lets the team test reasoning while reducing operational risk.

The agent can retrieve records, search documents, classify cases, propose actions, and explain what it would do.

It cannot yet modify the CRM, issue a payment, delete a record, send an external message, or change an account.

This phase answers several questions cheaply:

• Does retrieval return the right evidence?
• Does the model interpret it correctly?
• Does the agent choose the right tool?
• Does it know when evidence is insufficient?
• Does it route approval cases correctly?
• Does it terminate at the right point?

Tools are the model's interface to software.

Narrow tools are easier to understand and test than broad interfaces.

run_database_query(sql)

gives the model a large action surface.

get_customer_invoices(customer_id, start_date, end_date)

expresses a bounded operation.

Every write tool should define its accepted arguments, permission checks, possible errors, idempotency behavior, success response, and verification method.

A useful sequence is:

Authority should follow measured behavior and consequence.

A production trace should let an engineer reconstruct what happened.

Record the relevant input, model and prompt version, retrieved evidence, tool selection, tool arguments, tool result, state transitions, approvals, final output, resulting environment state, latency, and cost.

Trace data should respect privacy and data-handling rules. The goal is diagnostic visibility with selective logging.

A final answer can be correct after a poor execution path.

The system can retrieve an irrelevant policy, choose the wrong tool, recover by chance, and still produce the expected message.

A layered evaluation catches this.

Fix the layer that caused the failure.

The 2026 production study shows deployed systems favoring controllability over long autonomous trajectories. Treat it as context for architecture decisions.

Charts based on survey findings identify their source in the caption. Mathematical examples are labeled as illustrations. Architecture diagrams and decision frameworks are original explanatory assets created for this post.