Most AI agent demos work. Most AI agents in production do not, at least not at first. The gap is rarely the model. It is everything around it: what the agent is allowed to do, what it can see, what happens when it is wrong, and what it costs when it is busy.
I build agents and AI features that run inside real products, including FindRE, a CRM where AI writes call summaries, drafts replies and works the pipeline through an MCP server. This is the approach I use.
1. Start with one job, written down
An agent with a vague brief behaves vaguely. Before any code, write three things:
- The job. One sentence. "After each call, write a summary and propose the next step."
- What good looks like. Five to ten real examples of input and the output you would accept.
- What it must never do. Send anything to a customer without approval. Delete records. Spend more than a set amount.
If you cannot write the job in one sentence, you have more than one agent, or you do not need an agent at all. A fixed workflow with one AI step is cheaper, faster and easier to trust whenever the steps do not change.
2. Know the parts
Every agent, whatever it is built with, is the same six parts:
| Part | What it is | Where it goes wrong |
|---|---|---|
| Model | The language model doing the reasoning | Too small for the task, or far bigger than it needs |
| Instructions | The system prompt: role, rules, tone | Long, contradictory, never updated |
| Tools | Functions the model can call | Too many, badly named, return too much |
| Loop | Call model, run tool, feed result back, repeat | No step limit, no timeout |
| Context | What the agent knows right now | Everything stuffed in up front |
| Guardrails | Permissions, approvals, limits, logs | Added after the first incident |
Anthropic’s engineering note on building effective agents draws the same line I do: start with the simplest thing that works, and only add an agent loop when the task needs one.
3. Design the tools before the prompt
Tools decide what an agent can do, so they deserve more care than the prompt gets. Both main model families document the mechanism: tool use in Claude and function calling in the OpenAI API. If several assistants need the same tools, the Model Context Protocol is the standard way to publish them once.
Few and distinct
A model chooses well between six clear tools and badly between thirty similar ones. Merge tools that overlap.
Named for what they do
find_stalled_dealsbeatsquery_v2. The name and the one-line description are the documentation the model reads every single time.Small, structured results
Return the fields that answer the question, not the whole record. Large results cost money, slow the loop and bury the useful part.
Errors the model can act on
"No contact found with that email. Try searching by phone." lets the agent recover. A stack trace does not.
If several assistants or products need the same tools, put them behind an MCP server so they are built once. I cover that in how to build an MCP server.
4. Give it less power than you think it needs
The safest agent is one that cannot do the dangerous thing, not one that has been asked nicely not to.
- Permissions live in the tools. Each tool checks who the agent is acting for and what that person may see. Never trust the model to respect a boundary.
- Reading is free, writing is reviewed. Let the agent look things up on its own. Put a person in front of anything that sends, deletes, pays or publishes.
- Drafts over actions. In FindRE the AI drafts the reply and a person sends it. The time saving is nearly the same, and the risk is a fraction.
5. Fetch context, do not stuff it
It is tempting to paste everything the agent might need into the prompt. That gets expensive and makes answers worse, because the important part drowns.
Give the agent a small, stable set of instructions, and let it fetch the rest through tools at the moment it needs it: this lead's history, that call's transcript, the current pipeline stage. The context stays short, current and relevant.
6. Test on real examples before launch
This is the step that separates a demo from a product. Collect real cases, including the awkward ones: the angry message, the half-empty record, the call where nothing was decided. Run the agent against all of them and check three things:
- Did it pick the right tool?
- Is the output something a person would accept?
- Did it stay inside its limits?
Keep those cases. Every time you change a prompt, a tool or a model, run them again. Without that, every improvement is a gamble.
7. Control cost and failure from the first version
- Use the smallest model that passes your tests for routine steps, and a stronger one only where it matters.
- Stream responses so people see progress immediately.
- Cap everything: steps per run, tokens per run, spend per customer per day.
- Set timeouts and a fallback model, so one provider's bad hour does not become your outage.
- Track cost per feature from the start. It is the only way to price the product sensibly.
8. Log every step
For each run, record the input, every tool call with its arguments and result, the final output, the cost and the time taken. When someone asks "why did it do that?", and they will, the log is the answer. It is also where your next test cases come from.
What this looks like in a real product
In FindRE, the agent work is deliberately narrow. After a call, AI writes the summary and proposes a next step. When a message arrives, it drafts a reply for a person to review. An MCP server exposes the CRM as a small set of permissioned tools, so an AI assistant can work with leads and deals on request without touching the database directly. Nothing about it is exotic. It works because each piece has one job, limited power and a person where it counts.
If you want an agent like this built into your own product or operations, that is what I do under AI agents.