Guide · Agentic AI
Agentic AI architecture: multi-agent systems that survive production
Short answer: a production agentic system is a small number of agents with narrow jobs, a topology you can dry-run without a model, tools exposed through one standard (MCP), and an evaluation set that runs on every change. Most agent projects fail on handoffs, unbounded loops and token cost, not on model quality.
What is agentic AI?
Agentic AI is software where a language model decides which step to take next: it plans, calls tools or other agents, reads the result and continues until the goal is met. The difference from a chatbot is that the model chooses actions, not only words. That freedom is the value and the risk: every decision the model makes is a place the system can go wrong.
When should I use a multi-agent system instead of one agent?
Split into several agents only when the jobs need different tools, permissions or context that would crowd one prompt. Every extra agent adds a handoff that can fail, so start with one agent and split when evaluation shows it confusing tasks.
When I designed an 11-agent GenAI ecosystem for a global telco, the agent count came from the business, not the framework: billing, tariffs, device support and account changes each had their own data, rules and approvals. The work that took longest was not the prompts. It was defining cross-agent communication (who may call whom, with what payload), MCP integration for shared tools, and telemetry so every handoff can be traced.
The five parts of a production agent architecture
- Topology. A written map of agents, their single responsibility, and every allowed route between them, including the fallback to a human. If you cannot draw it, you cannot test it.
- Tool layer. Tools exposed once, through the Model Context Protocol, with least-privilege permissions per agent and logged inputs and outputs.
- Deterministic rails. Business rules, authentication, payments and anything regulated stay in code. The model chooses among safe actions; it does not invent them.
- Evaluation. A fixed set of real user goals, scored for task success, correct tool calls, cost and latency, run on every prompt, model or tool change.
- Budgets. Hard caps on steps, tokens and time per request, so a confused agent stops and escalates instead of looping on your cloud bill.
How do you test an AI agent before production?
Dry-run the topology without a model to prove every route and handoff exists, then run a fixed evaluation set of real user goals against the live agents, scoring task success, tool-call correctness, cost and latency, plus red-team cases for jailbreaks and PII leaks. Re-run the same set on every change.
I built The Matrix, part of The Caliper, for the first step: it maps a multi-agent design and dry-runs every scenario with no AI calls, so broken routes show up before anyone pays for inference. The Anvil covers red-teaming of the prompts and logic boundaries.
What does MCP do in an agentic system?
The Model Context Protocol (MCP) gives agents one standard way to discover and call tools and data sources. It replaces one-off integrations per agent, so a tool is built once and every agent and assistant can use it with the same permissions and logging.
I use MCP in my own tools too: Lumen CodeSearch is an offline MCP server that gives coding agents such as Claude Code, Copilot and Cline the exact lines they need, so the frontier model reads far less context.
Why agentic AI projects fail
- Too many agents too early. Each handoff loses context and adds latency.
- No stop condition. Agents retry and re-plan until the token budget, not the task, ends the run.
- Rules in prompts. A refund limit written in a prompt is a suggestion; written in code it is a rule.
- No regression set. A prompt fix for one market silently breaks another.
- Unpriced design. Nobody multiplied steps × tokens × traffic before launch. Crucible does that maths up front.
Frequently asked questions
What is agentic AI?
Agentic AI is software where a language model decides which step to take next: it plans, calls tools or other agents, reads the result and continues until the goal is met. The difference from a chatbot is that the model chooses actions, not only words.
When should I use a multi-agent system instead of one agent?
Split into several agents only when the jobs need different tools, permissions or context that would crowd one prompt. Every extra agent adds a handoff that can fail, so start with one agent and split when evaluation shows it confusing tasks.
How do you test an AI agent before production?
Dry-run the topology without a model to prove every route and handoff exists, then run a fixed evaluation set of real user goals against the live agents, scoring task success, tool-call correctness, cost and latency, plus red-team cases for jailbreaks and PII leaks. Re-run the same set on every change.
What does MCP do in an agentic system?
The Model Context Protocol (MCP) gives agents one standard way to discover and call tools and data sources. It replaces one-off integrations per agent, so a tool is built once and every agent and assistant can use it with the same permissions and logging.
Who can audit or design an agentic AI architecture?
Shashikant Jha, a Conversational AI Architect based in Pune, India, audits and designs agentic and multi-agent architectures for enterprise teams. He designed an 11-agent GenAI ecosystem with MCP integration for a global telco and offers consultations at shashikantjha.com.
Work with me
I audit agentic and multi-agent architectures, design lean agent topologies, and set up the evaluation and governance pipelines that let teams change them safely. Sessions run 15 to 60 minutes.
Related guides: Production GenAI: RAG, evaluation and safety · Conversational AI architecture and conversation design