Generative AI or deterministic rules: when to use an LLM
An LLM can read, summarise and write where a rule fails. A rule guarantees the same result on every run, which no LLM promises. The real question is not which side to pick, but which part of a process to give to each.
Published September 22, 2026
Two tools that do not solve the same problem
A deterministic rule is code or a rules engine: for the same input, it always produces the same output. It can be tested exhaustively, audited line by line, and every decision can be explained. Its limit is well known: it only handles what was anticipated. A letter phrased differently, a badly filled field, an attachment in an unexpected format, and the rule fails or, worse, gets it wrong silently.
A large language model (LLM) works the other way round. It understands free text, extracts information from a poorly structured document, and writes a reply that fits the context. But its output is probabilistic: two runs can give two wordings, sometimes two different conclusions. It can also produce a plausible, wrong answer without flagging it.
Same input, same output. Testable, auditable, explainable. Blind to anything that was not anticipated.
Understands natural language and messy documents. Variable output, possible and silent errors.
Pitting one against the other is like comparing a screwdriver and a scanner. The right question is: in my process, which step needs to understand, and which step needs to guarantee?
When a deterministic rule is enough, and wins
Some processes have nothing to gain from an LLM, and a lot to lose:
- Calculations: an amount, a due date, a rate, a balance. An LLM can get an addition wrong; a line of code cannot.
- Compliance checks: a regulatory threshold, an authorisation, a deadline. The answer must be the same for everyone and defensible in front of an auditor.
- Structured inputs: a form with defined fields, a file with a stable format, an API call. The data is already clean, there is nothing to interpret.
- High-stakes decisions: granting a right, approving a payment, deleting data. You want to be able to replay the decision and get the same result.
In these cases, an LLM adds inference cost, latency and a risk of error, with no benefit. If the rule is simple to write, write the rule.
When an LLM brings what no rule can
Conversely, the LLM becomes relevant as soon as the input is human language or a non-standard document:
- Reading and sorting emails, tickets and complaints written in free form.
- Extracting information from a contract, an invoice or meeting notes whose layout varies.
- Summarising a long document or a history of exchanges.
- Drafting a reply, a draft or a summary suited to its reader.
- Searching by meaning in a document base, rather than by exact keywords.
Trying to cover these cases with rules produces endless decision trees that are fragile and break on the first unexpected case.
The right pattern: the LLM proposes, a rule validates
In most business processes, the two are combined. The LLM handles what is ambiguous and produces a structured proposal; a deterministic layer checks it before any action.
- The LLM interpretsIt reads the request, extracts what matters and proposes an action: create a ticket, reply to a customer, update a record.
- A rule checksIs the format valid? Are the amounts within bounds? Is the user allowed to take this action?
- A person approves when the action commitsAny action that writes, sends or deletes is paused until explicitly confirmed.
- Everything is loggedThe proposal, the check and the human decision are recorded, so they can be replayed and explained.
This split has a decisive advantage: the probabilistic part never has the last word. A model error is stopped by the check or by the person approving, instead of spreading through your information system. It is also what makes the whole defensible in front of an auditor: the decisions that commit rest on written rules and logged approvals.
Human approval is only worth something if the person sees exactly what will be executed: the recipient, the content, the data being changed. Approving a vague summary of the action amounts to approving nothing.
A grid to decide, step by step
Rather than deciding for a whole process, examine each step with four questions:
| Question | If yes | If no |
|---|---|---|
| Is the input free text or a non-standard document? | LLM to interpret | Rule or code |
| Must the result be identical on every run? | Rule, at least to check | LLM possible |
| Is an error hard to undo? | Deterministic check and human approval | Check after the fact possible |
| Must the decision be justified to a third party? | Written rule and decision log | Simple log |
A single chain often contains both answers: an LLM to read a refund request written by a customer, a rule to check the ceiling, human approval above a given amount, code to execute the transfer.
The most common mistakes
Giving a calculation or a check to the model
Asking an LLM to verify that an amount respects a ceiling means accepting that it will sometimes be wrong. The model can extract the amount; the comparison with the ceiling must stay in code.
Letting the model act without a safeguard
An agent that can send an email or change a record without any check turns every misunderstanding into an incident. Actions that write must go through approval.
Writing rules for all of natural language
Conversely, trying to cover everything with rules leads to hundreds of special cases that nobody can maintain. If the input is language, let the model interpret it and check its output.
Judging on a demo
An LLM impresses on ten hand-picked examples. Measure it on a representative sample of your real cases, edge cases included, before deciding where it steps in.
Frequently asked questions
Can an LLM replace a rules engine?
No, not for decisions that must be reproducible and justifiable. It can, however, feed a rules engine by turning free text into structured data the rules know how to handle.
Can an LLM be made deterministic?
You can reduce the variability of its answers through configuration and an enforced output format, but that guarantees neither the same answer every time nor a correct one. The guarantee comes from a deterministic check placed after the model.
Does every action need human approval?
No. Reading, searching and drafting can run freely. Approval is justified for actions that write, send or delete, because their effects are hard to undo.
Where SmartAGT fits
SmartAGT applies this pattern by design. Agents read, search and draft freely; any action that writes or executes in a connected tool is paused deterministically until a person confirms or rejects it.
The operator sees exactly what will be executed before deciding, and every decision goes into a verifiable audit chain. Details are on the Security page.
