AI agent
Autonomous system based on a language model that executes multi-step processes on real systems. Unlike a conversational chatbot, an agent reads, decides, invokes tools (APIs), evaluates results and chains actions until completing an objective. In enterprise production it requires a 6-layer architecture: model, memory, tools, orchestration, observability and security.
What an AI agent is (and isn't)
An AI agent is a system that uses a language model (LLM) as the brain to make decisions and execute actions on real systems — APIs, databases, external tools — with the goal of completing a multi-step task. The key difference vs a chatbot is the ability to act, not just respond.
A conversational chatbot answers "what's my account balance?" with text. An agent reads the question, calls the CRM API to check balance, verifies exposure limits, reviews pending operations, drafts a personalised email if it detects anomaly, and updates the system with the interaction — all without human intervention between steps.
The difference isn't marketing. It's architecture: a chatbot lives in the conversation layer; an agent lives in the operations layer. And each solves different problems.
Rule of thumb: if the process involves more than 3 sequential steps, requires querying different systems or makes decisions based on data not in the prompt, it's an agent. If it's pure Q&A over known information, it's a chatbot (or better, RAG).
When to use an agent vs a workflow
Not every process wants to be an agent. Agents are expensive to run and hard to debug. A deterministic workflow with n8n or pure code is cheaper, faster and more reliable when the process allows.
The correct architecture is usually hybrid: an n8n workflow orchestrates the process at high level (business rules, timeouts, retries, escalation), and calls an agent only on steps requiring decision with ambiguity. This approach reduces operating cost by 40-70% vs "agent for everything".
| Situation | Best option | Why |
|---|---|---|
| 100 % fixed known rules process | n8n workflow or script | No ambiguity means AI adds no value. Operating cost doesn't pay off. |
| Natural language input process | Agent | The LLM interprets intent and decides the flow. A workflow would need 100 if/else. |
| 20+ step process with intermediate decisions | Agent | The agent keeps context between steps. Chained workflow breaks on debugging. |
| Must dynamically query multiple systems | Agent with tools | Agent decides which API to call per context. Workflow requires pre-designed flow. |
| High volume and critical latency (<1s) | Workflow + rule | Agent adds 2-5s latency per step. Not feasible in real time. |
| On-premise regulated data + high complexity | Local open-source agent | Llama or Mistral in your infra. Absorbs complexity without exposing data. |
Architecture: the 6-layer stack
An agent in production is not an LLM endpoint with a pretty system prompt. It's a 6-layer architecture, each with its own operational problems.
- Layer 1 · Model — Base LLM, tiered by difficulty. Don't use your most expensive model for everything: route Haiku/Flash for routine, Opus/GPT only for complex steps. Typical saving 40-60% without quality loss.
- Layer 2 · Memory — Persistent context + selective RAG. An agent without memory repeats work. Cramming all history into the prompt is expensive and noisy — retrieve only what's relevant for this iteration.
- Layer 3 · Tools — APIs and functions the agent can call. Explicit names, short descriptions, strict input schemas. A poorly defined tool breaks everything on top.
- Layer 4 · Orchestration — Workflow, retries, queues, humans in the loop. The agent decides what to do; the orchestrator decides when, how many times and with what timeout. Without this layer, an agent in production is an experiment with real budget.
- Layer 5 · Observability — Per-step traces, metrics, structured logs. You need to see exactly which prompt called which tool with what result and how much it cost. Without this, debugging is guessing.
- Layer 6 · Security — Guardrails, PII, roles, audit. Before connecting to the real world, you must be able to answer: "what can and can't it do, with what data?".
If you're missing any of the 6 layers, you have a chatbot on steroids — not an agent. And in production it will break when edge cases appear, APIs change or prompt injection attempts occur.
Real use cases by area
Agents generate value when the process combines high volume + decision with ambiguity + access to real systems. These are areas where we see most impact in Spanish enterprise.
- Legal / firms — Contract review, due diligence in data rooms, clause extraction.
- Finance and accounting — Bank reconciliation, invoice processing, recurring reporting, expense analysis.
- Back-office operations — Vendor onboarding, incident management, standard contract generation, 5 well-scaling cases.
- HR and talent — CV filtering, async interviews, employee onboarding.
- Premium customer support — Query triage, lead qualification, first-level support with human escalation.
- Insurers · claims processing — Multi-channel claim intake, extraction with human validation, escalation by confidence.
- Health · scheduling and triage — 24/7 voicebot with clinical derivation, post-treatment follow-up.
Operating cost of an agent in production
In real pilots we've put in production during 2026, operating cost sits between €0.03 and €0.12 per resolved conversation — with Claude Opus 5 tiered with Haiku 4.5, prompt cache active and selective RAG. Range highly dependent on context: in agents with 20+ steps and heavy RAG the cost rises to €0.25-0.80 per conversation.
Typical breakdown of operating cost:
- 55-70 % · LLM (model + input and output tokens)
- 15-25 % · Tools and memory (APIs, embeddings, vector DB)
- 10-20 % · Observability and guardrails (traces, evals, moderation)
Three things double the cost without anyone warning you: (1) cramming all context into the prompt "just in case", (2) using the most expensive model for all steps, (3) not caching the system prompt. Without prompt caching, a 3,000-token system prompt is paid on every call — with cache, once every 5 min.
Common errors that break agents in production
- 1. Not versioning prompts and tools — a change in production without record breaks the agent and you don't know what changed. Treat prompts like code: commit, review, deploy.
- 2. Using the most expensive model for everything — Opus 5 for classifying intent is throwing money. Route with logic: Haiku/Flash for 70% of decisions, Opus only for complex reasoning.
- 3. Not caching the system prompt — 3,000 context tokens without cache paid on every call. With Anthropic prompt caching you drop cost 65-90% on routine steps.
- 4. Cramming all history into context — the agent performs worse with more tokens, not better. Selective RAG: retrieve only what's relevant for this iteration.
- 5. Poorly defined tools — ambiguous description or schema without restrictions. The agent calls tools inventing parameters. Strict Pydantic/Zod schemas are mandatory.
- 6. No per-step observability — you know the output is bad but not where it failed. Per-step traces with LangSmith, Langfuse or Helicone are the only way to debug.
- 7. No human in loop on irreversible steps — payments, client notifications, legal decisions. Never. Always human confirmation.
- 8. No cost limit per conversation — a bug can make an agent loop and consume hundreds of euros in tokens in minutes. Circuit breaker mandatory.
How to start: 4-step framework
The right way to start with agents is not a big 6-month project. It's an acotated 6-8 week pilot on a process with clear ROI. This framework is what we use with every new client:
- Step 1 · Pick the process — apply the 5-question framework (volume, stability, measurable, reversibility, sponsor). If it fails in 2 or more, it's not your first pilot.
- Step 2 · Test with base model — before full architecture, validate a base LLM + 2-3 tools can do the task. Weeks 1-2 in Jupyter with small dataset.
- Step 3 · Design the minimum 6 layers — tiered model, selective RAG, tools with schema, orchestrator with retries, per-step observability, basic guardrails. Weeks 3-5.
- Step 4 · Production with supervision — deploy in assisted mode (human reviews 100% of output) → gradual (50% → 20% → 5% → auto) with quality metrics. Weeks 6-8.
You can evaluate your process with the "What to automate first?" utility (5-question framework with 0-100 score) and estimate cost with the ROI calculator before asking for diagnosis.
Frequently asked questions
What's the real difference between a chatbot and an AI agent?
A chatbot lives in the conversation layer: receives text, returns text. An agent lives in the operations layer: receives an objective, decides what steps to execute, calls real systems (APIs, databases), evaluates results and chains actions until completing the task.
The key difference is the ability to act. A chatbot tells you "your invoice for 12/2025 is €234". An agent reads the invoice, compares it with history, detects an anomalous charge, cancels the charge if appropriate, notifies the client and updates the CRM — all without intermediate human.
Chatbots resolve informational queries. Agents execute operational processes. Require different architectures and have different operating cost (agents 3-10x more expensive per interaction, but absorb human work of another magnitude).
Which LLM model is best for agents in production?
Depends on the specific step and budget. For complex multi-step agents (20+ decisions), Claude Opus 5 is today the model that best keeps context without collapsing. GPT-5 performs similarly on many tasks and sometimes better on code generation. Gemini 2.5 Ultra has an advantage on RAG over very long documents (+1M token contexts).
For routine steps (classification, extraction, simple decisions) Claude Haiku 4.5 or Gemini 2.5 Flash are much cheaper and good enough. A well-designed agent routes 70% of decisions to small models and only fires frontier models when needed.
You can use our LLM model selector utility for a recommendation based on your specific case.
Can an agent be deployed fully on-premise for regulated data?
Yes. For clients in banking, health, insurance or public sector that can't take data out of their infrastructure, we deploy agents based on open-source models (Llama 3.3, Mistral Large, DeepSeek) on-premise or in European private cloud (Azure Spain, GCP Madrid, OVH).
Trade-off: open-source models today perform 5-15% below frontier commercial ones on complex agent tasks. For steps requiring maximum quality, hybrid architecture can be done: most decisions on-premise, and only critical steps delegate to a commercial model under enterprise no-retention contract (Anthropic Bedrock, Azure OpenAI, Vertex).
Operating cost of on-premise agents is different: you don't pay per token but pay infrastructure (GPUs), maintenance and updates. At high volume, on-premise is cheaper; at low volume, API is better.
How much does an agent in production cost per month?
Monthly operating cost of an enterprise agent typically sits between €200 and €5,000/month depending on volume, complexity and chosen model. A first-level customer support conversational agent with 5,000 conversations/month typically costs €300-800/month. An agent processing 2,000 documents/month with RAG over large base can be €1,500-3,500/month.
On top of that add infrastructure (vector DB, observability, orchestrator) which usually adds €150-400/month depending on chosen tools.
Pilot implementation cost is a separate one-off: €15-40k typically for an acotated pilot. The ROI calculator gives you a specific estimate for your case.
What AI governance is needed to put an agent in production under AI Act?
Depends on system risk category under AI Act. Agents used in processes affecting people (credit, HR, health, biometrics, critical infrastructure) are "high risk" and require technical file, conformity assessment, risk management system, transparency and human oversight.
For low or minimal risk agents (back-office, internal analysis, basic support) obligations are transparency and good practices, much lighter.
In any case, an agent in production requires: registration in company's AI inventory, use case documentation, data policies (GDPR + sectoral), continuous monitoring and human escalation plan if system fails.
See also our guide AI Act EU 2026: what a Spanish company must do.