A demo answers a question
A demo shows that a model can read a document, draft a reply, or summarize a meeting. That is useful to know. It is not the same as a system that runs every day inside your operation, on your data, under your rules, with someone accountable when it is wrong.
What production requires
Five things separate a production system from a pilot.
- Connection to your systems of record. The work lives in your EHR, ERP, core platform, or field service system. AI that sits beside those systems creates one more place to copy and paste.
- Decision rights. Someone owns each workflow, decides what the system may do on its own, and approves what it may not.
- A record. Every action leaves a trace: what the system saw, what it did, which model it used, and who approved it.
- Measurement. Value is counted in the measures leadership already uses, against a baseline taken before anything changed.
- Operation. Models, vendors, and workflows change. Someone watches the system, reviews its exceptions, and improves it.
Why pilots stall
MIT NANDA's 2025 report, The GenAI Divide, found that 60 percent of organizations evaluated enterprise AI tools, 20 percent reached a pilot, and 5 percent reached production.
Pilots are usually built to prove that a model works. They skip the five requirements because those are slow and unglamorous. When it is time to go live, the work that remains is larger than the work that was done, and the pilot stalls.
Start with the operation, not the model
We begin every engagement by mapping the workflow, the people who own it, the systems it touches, and the rules it must follow. Only then do we choose a model. The model is usually the easiest part to change later. The controls around it are not.
Three questions for leaders
Ask these of any AI initiative. If the answers are clear, the project is ready for production. If they are not, it is still a pilot.
- Where will it run, and under whose security controls?
- Who approves what it does?
- How will we know it is working, in numbers we already report?