Why enterprise AI breaks between demo and production

Most enterprise AI projects do not fail at the demo stage. The demo usually works. A small group chooses a clean example, the prompt is tuned for it, the model answers convincingly, and the room can see potential.

The failure happens later, when the system has to survive production conditions.

The demo hides the operating model

A demo proves that a model can produce a useful answer in a controlled situation. It does not prove that the company can operate the system.

Production requires answers to less exciting questions:

1. Who owns the workflow after launch?
2. Which data can the model see, and under whose permissions?
3. What happens when retrieval returns stale, conflicting, or sensitive context?
4. Which tool calls need human approval?
5. How will the team reproduce a bad answer or a risky action?
6. What is the cost per task under realistic usage?
7. Who updates prompts, evals, documents, and escalation rules?

If these questions are not answered, the project becomes a demo with operational debt.

The real unit is the workflow

Enterprise AI should start from a workflow, not from a model.

A useful workflow has pressure: it is slow, expensive, risky, or strategically important. It also has observable inputs, outputs, decisions, users, permissions, exceptions, and success metrics.

When teams start from the model, they ask:

What can we use this model for?

When teams start from the workflow, they ask:

Which part of this process can be improved without creating unacceptable risk?

The second question is much more useful.

Context is not just a longer prompt

Many production failures are context failures.

The model receives too much context, stale context, context from the wrong user role, context that should have been filtered before retrieval, or tool output that is treated as trusted instruction.

For production systems, context needs architecture:

• retrieval scoped by permissions;
• separation between task context, business context, user context, tool context, and memory;
• logs of the actual context package sent to the model;
• evals for retrieval quality and not only final answer quality;
• rules for what should never enter the prompt.

This is why context engineering matters more than prompt decoration.

Evals are the bridge to production

If there are no evals, every change is a guess.

A serious pilot needs a small but real evaluation set:

• examples from the actual business process;
• expected outcomes and unacceptable outcomes;
• adversarial cases;
• retrieval checks;
• tool-call checks;
• refusal and escalation checks;
• cost and latency measurements.

Accuracy alone is not enough. The system must be checked for security, cost, observability, and operational behavior.

Tool-using agents change the risk model

A chatbot can be wrong. A tool-using agent can be wrong and act.

This changes the security model. You now need to review:

• which tools exist;
• what each tool can do;
• what identity the agent uses;
• which actions have side effects;
• which tool outputs are untrusted;
• which actions require confirmation;
• how all important model and tool calls are logged.

Prompt injection is only one part of the problem. The bigger question is whether the system stays safe when hostile or ambiguous context enters the loop.

The production checklist

Before moving an AI pilot forward, I want to see evidence in six areas:

1. Business pressure: the workflow matters enough to justify the system.
2. Data readiness: the relevant knowledge exists, is accessible, and can be permissioned.
3. Architecture: context, tools, memory, and handoff are explicit.
4. Evals: the team can measure regressions and risky behavior.
5. Security: data leakage, tool misuse, prompt injection, and approvals are reviewed.
6. Ownership: someone owns prompts, evals, monitoring, costs, and updates after launch.

Without this, the safest decision is often not to build yet. Audit first, narrow the workflow, and define the smallest useful production slice.

That is less glamorous than another demo. It is also how AI systems become useful.