A Product Manager's Guide to Agent Workflows
Product managers do not need to become agent framework experts.
They do need to stop writing agent requirements like this:
Build an AI agent that helps with customer support.
That sentence is a fog machine.
A useful agent spec names the outcome, the boundaries, the tools, the approvals, and the metric.
Start with the job#
Write:
The agent drafts support replies for billing questions by reading account state, invoices, support history, and refund policy. It requires approval before promising credits or issuing refunds.
Now engineering has something real.
Define what the agent owns#
Every workflow needs an ownership line:
- What the agent decides.
- What the agent drafts.
- What the agent routes.
- What the agent cannot do.
- What a human must approve.
Without that, the scope will drift.
Tools are product requirements#
If the agent needs tools, say which ones:
- CRM.
- Support desk.
- Billing provider.
- Docs.
- Repo.
- Observability.
- Slack.
Also say whether access is read-only, write-approved, or forbidden.
Evals are acceptance criteria#
Agent acceptance criteria should include eval cases:
- Correct classification.
- Missing context.
- Conflicting policy.
- Sensitive data.
- Approval required.
- Low confidence.
- Tool failure.
This is better than "the answer should be good."
Launch metrics#
Good metrics:
- Time to first useful draft.
- Human edit rate.
- Correct routing rate.
- Source coverage.
- Approval rejection rate.
- Unsafe action blocked.
- Cost per useful run.
Bad metric:
- Number of AI messages sent.
More messages is not automatically more value.
Build it in Codelit#
Try this:
Create a product requirements workflow for an AI support agent. Include user outcome, agent responsibilities, tools, permissions, approval gates, eval acceptance criteria, launch metrics, architecture, and repo handoff.
The PM job is not to ask for an agent. It is to make the agent's job impossible to misunderstand.
Continue learning
Go deeper on AI agents and LLM systems
As an Amazon Associate I earn from qualifying purchases. Codelit may receive a commission at no extra cost to you.
The Plant Was Never the Problem
How Engineering Managers Help People Speak Up, Grow, and Ship
A practical guide to the management systems that help engineering teams surface problems and keep shipping.
View on AmazonThis book is written by Codelit founder Mo Sharif. Its Amazon link is a paid affiliate link.
AI Engineering
Chip Huyen · 2025
Book-length treatment of evals, prompts and deployment tradeoffs for foundation-model apps.
Agentic Design Patterns
Antonio Gulli · 2025
21 agent patterns with runnable code, so you can name the shape before you build it.
An Elegant Puzzle
Will Larson · 2019
Worked systems for team sizing, hiring loops, and migrations, drawn from Uber and Stripe.
4.5 (1.4k)