The demo worked. Someone wired a model to a spreadsheet of real questions, it answered most of them well, and the room agreed it was ready. The next sentence is always the same: let us connect it to the actual system.
That sentence is where the project changes shape, and almost nobody notices. A pilot produces text a person reads and judges. A connected model takes actions nobody reads first. The second is not the first with an API key added.
Four questions decide whether that connection is a good idea. None is technical. All four are the business’s to answer, and no amount of engineering covers the gap if it cannot.
A pilot and a connection are different projects
In a pilot the model is advisory: it drafts and suggests, a human reads every output, and the cost of being wrong is a few wasted seconds. Connected, it stops advising and starts doing — writing a record, sending a message, refunding an order — and nobody reads the output first, because the point of connecting it was that nobody had to.
This is the step where most projects quietly stall, and it is rarely the model. The four questions below have no owner, so nobody answers them and the work sits at ninety per cent for a quarter. That is the pattern behind where UAE AI pilots stop: every one of the four is a business decision wearing technical clothes.
1. What is it allowed to do without asking
Not what it can do. What it is allowed to do on its own, at three in the morning, with nobody watching.
The answer is not a policy document. It is two lists. First, everything the model may read: which tables, which documents, which inboxes. Second, every action it may take that changes something. List one is a privacy question. List two is the entire risk surface of the project, and it is shorter than people expect once written down.
Every line on the write list gets a label. Do it: the model acts alone. Propose it: it prepares the action and a person confirms with one click. Never: it does not get the capability at all.
Two rules make the list easier. Anything that moves money, sends something to a customer under your name, or deletes a record starts on propose. And anything you would not let a competent new employee do unsupervised in their first week does not go on do it either, because the model has less context than that employee, not more.
This failure has a name. The OWASP Top 10 for LLM Applications lists excessive agency among the ten: a system given more capability or permission than the job required, usually because the permission was convenient in development and nobody narrowed it before launch.
The test in the meeting: read the write list aloud and ask who is accountable if each line happens wrongly a hundred times overnight. A line with no name against it is not ready.
2. What does it see, and where does what it sees go
Every prompt is a disclosure. Whatever the model needs in order to answer, you have handed to whoever operates it, and often to their supplier too. So the question is not whether the provider is reputable but what specifically leaves your systems, and how far. Three parts, each with an answer that fits on a line:
- What is in the prompt. Names, phone numbers, order histories, a whole document because that was easier than extracting the paragraph.
- Where it goes. Which company, in which country, under which contract, retained for how long, and whether it can be used to improve anyone’s model.
- What comes back, and where it is stored. Outputs are records too, and they land in your database with everything else.
What makes this manageable is unglamorous: send the field, not the file. A model that needs to know an order is late does not need the customer’s full record. Trimming the payload cuts the privacy exposure and the running cost at once, and it is an afternoon of work.
The regulation is a constraint on that design, not the subject of it. Federal Decree-Law 45 of 2021 governs personal data here, and the obligations it creates before you connect anything are in what the PDPL asks of you first. The UAE Charter for the Development and Use of Artificial Intelligence names data privacy, transparency and human oversight among its principles. Neither tells you how to build. Both tell you which design options are not available.
3. How will you know when it is wrong
This is the question that separates a system from a demo, and the one most reliably skipped. Ask the team what today’s error rate is. Not whether it is good — what the number is. If nobody can answer, nothing is being measured, and the honest description is that it works until a customer says it does not.
Three things must exist before the connection goes live, all cheap against what they prevent:
- A set of examples with known right answers. Fifty to two hundred real cases, the awkward ones included, with the correct outcome written beside each. It is the only way to tell whether a prompt change, a model upgrade or a new supplier helped. Assemble it before launch, because afterwards nobody has time.
- A log of every input and every output. Not sampled, and each with a timestamp and a reference back to the record it touched. In month four this is the difference between an hour of investigation and a week.
- A human reading a sample, on a schedule. Twenty outputs a week, read by someone who knows what good looks like. An hour, and it catches drift long before a customer does.
If you want a vocabulary rather than inventing one, the NIST AI Risk Management Framework organises the work as govern, map, measure and manage. Its value at this size is not the paperwork: it is that measure is a named function with an owner, which is the thing that goes missing.
4. What happens when it fails on a Friday night
Everything connected to a live system eventually fails, and it fails when nobody is looking. Four answers, written down before launch:
- The fallback. When the model is unavailable or returns nonsense, what happens instead. Queue it, hand it to a person, show an honest message, or stop. Any of those is fine. No answer means the answer is whatever the code happens to do, usually a blank page.
- The switch. One control that turns the model off and leaves the rest of the business working. Someone non-technical must be able to use it, and it must be tested in daylight rather than discovered in an incident.
- The spend ceiling. A hard daily limit with an alert well below it. A retry loop is the classic way a modest monthly bill becomes a large one overnight, which is why OWASP lists unbounded consumption in the same ten.
- The name. Who is called. Not a team. A person, with a second name behind them. Everything above is theatre without this line.
None of this is exotic; it is what any other production dependency gets. It is easy to skip because the model felt like a feature during the pilot, and features do not get pagers. Once connected it is infrastructure, and deferring that is how a decision gets paid for twice.
One answer that makes all four cheaper
One design choice improves every answer above: make the boundary between the model and your system a contract rather than a conversation. The model does not return prose your code then interprets. It returns a defined structure with named fields and permitted values, and your system validates it before acting. Providers now enforce this at the point of generation rather than leaving you to hope, and the structured outputs documentation sets out what the guarantee covers.
This is not tidiness. A structure gives the model somewhere to put a refusal, so it can say it does not know instead of inventing something, and somewhere to attach a confidence value, which is how a propose action decides whether to ask a human. It also turns “the model was wrong” into a field-level assertion you can test, which is what makes question three answerable at all.
The related discipline is to add autonomy only where a simpler shape has been measured and found wanting. Anthropic’s write-up on building effective agents argues the same: start simple, evaluate, add multi-step autonomy only when the simpler system demonstrably underperforms. Most problems proposed as agents are one well-specified call with a validated output.
The order I would take them in
- Write the read list and the write list, and label every write line do it, propose it or never. An hour with the people who own the process, not the code.
- Trim the payload to the fields the task needs, and record where they travel and how long they are kept.
- Build the example set, before anyone tunes a prompt against a feeling.
- Define the output structure and validate it in code. Refuse to act on anything that fails validation.
- Turn on logging, the switch, the spend ceiling and the fallback. Test the switch.
- Launch the propose version only, and watch for a few weeks what people correct.
- Promote individual lines from propose to do it one at a time, on evidence from the log, never as a batch.
Steps one to five are two to three weeks for a business that has already run a successful pilot. That is the real gap between a demo everyone liked and a system the company can rely on: smaller than most teams fear, larger than most vendors imply.
If the pilot has convinced people and nobody is sure what happens next, that is the work I do as an AI consultant in the UAE, and it sits inside a fractional CTO engagement when the same questions apply across more than one system.
When the four answers are in place and the connection has to be built, my company builds it: AI agents and automation at Tothiq.
Frequently asked questions
Do we need all four answered before any AI reaches production?
Before any model takes an action on its own, yes. A read-only assistant that drafts text for a person to approve needs question two answered properly and the other three lightly. The moment it writes, sends or pays with no human in the path, all four apply — that is when mistakes stop being visible before they reach a customer.
How long does this take for a company with no AI team?
Two to three weeks alongside normal work, for one connected use case, once a pilot exists. Most of it is decisions rather than code: the two lists, the payload trim, the example set. Companies that shorten it skip the example set, then argue for far longer about whether a change helped.
Is it safer to use a large provider or to run a model ourselves?
Neither is safer by default. Running your own removes one data transfer and adds an operational burden that has to be staffed. A hosted provider gives you better models and a contract you can read. Decide on what is left in your prompts after the payload trim, because for many business tasks nothing sensitive is left at all.
What is a reasonable error rate to accept?
Whatever the process it replaces already produces, measured the same way. That comparison is almost never made, because the human error rate is rarely known either. A model right nineteen times in twenty may improve on a rushed inbox or regress against a validated form, and only the example set tells you which.
Who should own this inside the company?
The person who owns the process the model touches, not the one who built the integration. Support owns a support assistant; finance owns anything touching invoicing. The builder cannot both propose the autonomy and judge whether it is safe.