Every AI system we review has the same smell somewhere in it: a language model being asked to do a job that deterministic code does better, cheaper, and with a guarantee. Counting rows. Looking up a price. Sorting a list. Checking whether a date falls inside a range. The model will attempt all of these, confidently, and be wrong just often enough to erode trust in the whole product.
The rule we use is simple: if the answer is knowable without judgement, do not ask the model. Use the model where judgement is the work — interpreting an ambiguous request, drafting a reply, deciding which of three plausible next steps fits the context. Use code where the answer is a fact.
This is not a purity argument. It is a reliability and cost argument. A database lookup costs fractions of a cent and returns the same answer every time. A model call costs real money, adds latency, and is usually right. "Usually right" is fine for a draft email. It is not fine for an invoice total.
The split most teams get wrong
The common failure is a prompt that quietly absorbs business logic. "If the customer is on the enterprise plan and the ticket is older than 48 hours, escalate." That is a rule, not a judgement call, and it belongs in code before the model ever sees the ticket. When rules live inside prompts, they cannot be tested, versioned, or audited like code — and they drift every time someone edits the prompt for a different reason.
The fix is a hard boundary: code decides the facts, the model decides the words. Fetch the account status, the order history, and the policy from your systems first. Then hand the model the facts plus the request, and let it do what it is good at — composing, summarising, interpreting tone.
A practical test: if you can write the correct behaviour as a unit test with a single expected output, it should not be in the prompt. If the correct behaviour depends on reading between the lines, it probably should be.
Where the model earns its keep
None of this means shrinking the model's role. The model should handle the genuinely ambiguous parts: a customer message that could be a complaint or a question, a document that needs summarising in the customer's own terms, a set of options that need ranking by fit rather than by price.
The pattern that works is a pipeline, not a monolith. Code gathers and validates. The model interprets and drafts. Code verifies the output — format, numbers, policy constraints — before anything reaches the customer. Each stage does what it is built for, and each stage is testable on its own.
We help companies design AI systems where the model does the thinking and the code does the counting — because a system that asks the model to look up a price will eventually quote the wrong one to the wrong customer, and no amount of prompt engineering fixes a job that was never the model's.
Field note: In one support-automation build, the agent was asked to calculate refund amounts from order history in the prompt. It was right about 96% of the time — which meant roughly one wrong refund in every twenty-five. Moving the calculation into code and letting the model write only the explanation took the error rate to zero and cut the token cost per ticket by a third. The model got better at its real job the moment it stopped doing the database's.
Related service AI products



