Who Holds the Decision Rules? Dividing Work Between Rules Engines, DMN, Policy Code and LLMs

Why 'who holds the decision rules' matters

This is part 2 of the series 'AI That Decides: The Promise and Limits of Decision RAG'. Part 1 framed Decision RAG as a proposal to output decisions such as 'approve, reject, assign', together with their grounds, in structured form based on retrieved evidence, and left open the question of 'how much to hand over to the LLM'. This part answers that question with a practical methodology.

The issue is where the decision logic lives. When an LLM is given the decision, the decision criteria are usually written into the prompt as sentences. A study on DMN-guided prompting paper points out that when decision logic is buried in prompts this way, business users find it hard to fix or refine, and prompt writing itself comes to rely on trial and error, creating problems for transparency, reuse and maintenance. The probabilistic nature of generation, where the same input can yield different answers, is also a burden for decisions. When an auditor asks 'why was this case approved?', 'the model decided so' is not an answer.

That said, a rules engine alone is not enough. A rules engine runs only when structured values (amounts, dates, scores) come in, but real-world inputs are often unstructured documents such as invoices, medical certificates and emails. So it helps to view a decision as two jobs. Compare it to a clerk who organizes the facts and an examiner who consults the rulebook. The clerk pulls 'the claim amount is X, enrolled for Y months' out of messy paperwork and organizes it, and the examiner checks those facts against the rulebook and reaches a conclusion. The question of this note is, in the end, who to seat in the clerk's chair and who in the examiner's.

An old solution: business rules management systems and DMN decision tables

The idea of separating decision logic from program code and managing it on its own is not new. It is what business rules management systems (BRMS) have long done. For example, the IBM Operational Decision Manager product page source describes the product as used to discover, capture, analyze, automate and govern rule-based business decisions. It also states that it documents how decisions were made to support audits and compliance, and allows policies to be changed quickly when regulations change.

DMN (Decision Model and Notation) is a standard for depicting such decision logic. The official OMG page source introduces DMN as a modeling language and notation for precisely specifying business decisions and business rules, and explains that it was designed to be used together with BPMN, the process standard. According to the OMG specification page source, DMN consists of two tools: a graphical notation showing dependencies between decisions, and an expression language detailed enough to be executable. It also has an XML representation, so decision models can be exchanged between organizations. The page lists official versions from 1.0 (September 2015) to 1.6 (September 2026), plus a 1.7 beta. The Camunda DMN tutorial source names the core elements as the decision table (inputs, conditions, outputs), FEEL (Friendly Enough Expression Language) for writing conditions, and the decision requirements diagram (DRD), used when a decision cannot be handled by a single table.

Let's build one. Below is a hypothetical decision table for deciding automatic payment of small insurance claims. Assume rows are checked from the top and the first matching rule is applied.

| # | Documents complete | Claim amount (KRW) | Enrollment period (months) | Claims in past year | Action | |---|---|---|---|---|---| | 1 | false | - | - | - | "Request more documents" | | 2 | true | < 1000000 | >= 12 | <= 2 | "Automatic payment" | | 3 | true | < 1000000 | < 12 | - | "Staff review" | | 4 | - | - | - | - | "Staff review" |

Reading it is simple. Each row is one rule, and all conditions within a row are joined by AND. '-' means 'any value' source. Row 2 reads: 'If the documents are complete, the amount is under 1,000,000 won, enrollment is 12 months or more, and there have been 2 or fewer claims in the past year, pay automatically.' Any case that escapes all conditions is caught by the last row and handed to a person. Conditions such as `< 1000000` and `>= 12` are FEEL expressions. The Camunda tutorial source explains that FEEL can express conditions such as string matching, true/false, numeric comparisons and ranges, and whether a date is before or after another. The hypothetical loan table used as an example in the DMN-guided prompting study paper ('approve if annual salary exceeds 50,000 and credit score is 700 or higher', with a last row of '-' catching everything else) has the same shape. The key point is that the engine executes the same table that business users read. Camunda explains that this reduces misunderstandings between analysts and developers and allows quick changes even in production source.

Policy as code: OPA and decision queries

If DMN started from tables read by business users, the 'policy as code' movement came from the development and infrastructure side. The official OPA documentation source introduces OPA (Open Policy Agent) as an open-source, general-purpose policy engine that unifies policy enforcement across the stack. Policies are written in a declarative language called Rego. Declarative means stating 'which conditions must be true' rather than 'compute in this order'. The documentation lists microservices, Kubernetes, CI/CD pipelines and API gateways as use cases, and notes that OPA is a CNCF graduated project.

The core of the design is separating decision-making from enforcement source. When a service needs a decision, it queries OPA with structured data such as JSON as input, and OPA evaluates that input against policies and data and returns a decision. Enforcement remains the service's job. It also matters that a decision result is not limited to yes/no such as allow/deny but can be arbitrary structured data. The documentation's example policies are 'internet-facing servers must not expose the insecure http protocol' and 'telnet must not be exposed'. Once these conditions are written in Rego, they can block such servers at creation time, or let the compliance team periodically find servers in violation.

When combined with LLMs, the lesson of OPA-style design becomes clear. The decision engine only needs to honor the contract 'structured input in, structured decision out', and it does not care who produced the input. If an LLM passes facts extracted from unstructured documents in a fixed JSON shape, the decision can be handled by policy code that people review and manage with versions. Compared with DMN, the difference is who writes it. Decisions that business users must read and edit themselves suit DMN decision tables, while decisions managed by engineers, such as access permissions and deployment conditions, suit policy code like Rego.

Comparing three division-of-work patterns

Ways of mixing LLMs with deterministic rules fall broadly into three. First, LLM-only decisions: rules are written in the prompt and the LLM also produces the decision. Second, LLM extraction + rules engine decision: the LLM only extracts facts from unstructured documents and passes them in a fixed schema, and a decision table or policy code makes the decision. Third, rules engine decision + LLM assistance: the input is already structured, so the engine makes the decision, and the LLM only helps by writing explanations to send to customers or reviewing exception cases that the rules routed to 'staff review'.

Here is how they compare by criterion (the assessments in the table below are this note's own judgment). Accuracy: LLM-only can waver when applying rules over several steps (see the research in the next section). The extraction pattern makes accurate decisions in themselves, but if extraction is wrong, the decision is wrong too. The rule-decision pattern produces correct decisions when the input is correct. Consistency: only the two patterns that include a rules engine guarantee the same decision for the same input. Auditability: a rules engine can record 'rule number N was triggered', but with LLM-only there is no guarantee that the explanation the model writes matches the actual reason for the decision. Cost of changing rules: editing one line of a prompt looks easy, but its impact is hard to verify. On the rules engine side, the change process is heavier, but the effect of a change can be tested table by table. Unstructured input: the rule-decision pattern can be used only when the input is structured.

So the suitable tasks differ. LLM-only fits initial triage or drafting where risk is low and rules still exist only as prose and change often. The extraction pattern fits tasks where input is unstructured but criteria are written down, such as reviewing insurance claim documents or loan application documents. The rule-decision pattern fits tasks where inputs such as amounts and scores are already in a database and audit requirements are strict. In practice, the second and third patterns are often used together within a single task: the LLM extracts facts, the engine decides, and the LLM explains the decision. In that case, the explanation should be written using the engine's decision and the number of the rule it was based on as input, so that the decision and the explanation do not diverge.

Where research is heading

All figures below come from the experimental conditions (datasets, models, tasks) each paper set. They should not be read as meaning the same effect will appear in other kinds of work.

DMN-guided prompting paper models the decision logic in DMN and uses that model to break the prompt structure into fine-grained parts so the LLM follows a defined decision path. It was applied to generating feedback on student assignments in a graduate course, and instructors evaluated the generated feedback by labeling it correct or incorrect. In this case study, results were better than when the same content was written only as a CoT (Chain-of-Thought) prompt, and in a survey based on the Technology Acceptance Model (TAM), students said the feedback was useful. The LLM still makes the decision, but because the source of the rules lives in DMN, which business staff can edit, the approach sits between "LLM alone" and "rule engine."

The symbolic working memory study paper looked into why LLMs miss rules. In a preliminary analysis, LLMs were strong at applying a rule once, but performance dropped as the number of steps grew, and got worse when rules were given in an order different from the order of application. The researchers identify "rule grounding"—finding which rule to apply to which fact at the current step—as the cause. So they store facts and rules in an external working memory in two forms, natural language and symbols (Prolog), and split the work: grounding is done by symbolic matching, and the actual rule application by the LLM. On four datasets—CLUTRR, ProofWriter, AR-LSAT and Boxes—they reported better results with GPT-4 and GPT-3.5 than CoT-style and symbolic baselines. The neurosymbolic representation study paper takes a slightly different direction. It converts the model's Hidden Representation into neurosymbolic vectors, solves the problem in that space, and then merges the result back into the original state. On a set of math reasoning tasks, it reported on average 88.6% lower cross-entropy loss and 15.4 times more problems solved than CoT prompting and LoRA fine-tuning. However, since this study deals with numerical calculation rules rather than business rules, it is better read as evidence pointing the same way—"precise rule-following is better handed to the symbolic side"—than as something to carry directly into a decision system.

Policy-as-Skill paper is the closest to practice. It is a runtime that bundles evidence verification, review routing, version control and auditing into "executable, versioned policy functions." With a fixed Gemma4 backend, it compared 13 methods on 600 development tasks. PaS+Audit achieved 53.8% exact match, a review F1 of 0.854, and 1.000 for both citation precision and audit completeness, outperforming LLM+RAG on most governance and review metrics. Adding deterministic controls raised overall accuracy to 61.2%, but the effect varied widely by task. On this basis, the researchers conclude that rule-based interventions should be applied selectively rather than wrapped around every decision. It is also worth remembering that exact match is only slightly above half.

How commercial tools work, and where the ontology fits

Commercial tools follow a similar division of labor. This is not a claim that one product is better; it is a look at the designs described in official documentation. The IBM ODM product page source highlights an interface for authoring and testing decisions, testing and simulation against scenarios and key performance indicators (KPIs), role-based access management, and governance and change management features. It also introduces an open-source-based product that follows DMN (IBM Decision Manager Open Edition). It further explains that integrating with the conversational features of watsonx Orchestrate lets users call the decision engine through conversation, which resembles the third pattern: the LLM handles the conversation and the engine makes the decision.

Palantir's AIP Logic documentation source describes it as a no-code environment for building, testing and deploying LLM-based functions. Logic functions take ontology objects or text as input and can return objects or strings, or edit the ontology directly. As a use case, it cites linking key information from unstructured input to the ontology. When automation is set up, ontology edits can be applied immediately or sent to a queue for human review. The same documentation states that row- and column-level access controls only filter the data the model "reads." Because these controls do not extend to the model's output or to ontology edits the model makes, it recommends protecting downstream data by also using marking- or classification-based access controls.

This is where the ontology series connects. An action type in an ontology is where you define in advance "which objects can be changed, under what conditions, and how." This is exactly the gate that turns a decision into actual execution. Even if the rule engine or LLM issues a decision of "automatic payout," changing the payout status must go through an action type with permissions and validation attached. This structure, which separates decision from execution, follows the same philosophy as OPA's separation of decision and enforcement. Even if a decision is wrong, the execution gate can stop it once more, and the execution side can decide, based on risk, whether to apply it immediately or route it to human review.

When rules change: version control and regression testing

Rules will change. Regulations are revised, automatic payout limits go up, and exception clauses get added. The steps to follow at that point can be summarized as follows. First, take the decision rules out of the prompt text and move them into a decision table or policy file. DMN has an XML interchange format source, so decision models can be handled as files, and Rego policies are code files too. Second, version the rules. This is the same idea as Policy-as-Skill paper bundling policies into "versioned executable units." Third, for each decision result, record which rule version and which rule number were applied. Only then can you later reproduce "that decision at that time" under the same conditions.

Fourth, build a regression test set: a collection of representative cases pairing inputs with expected decisions. For the insurance decision table above, add the boundaries of each row one by one (exactly 12 months, just below and just above 1 million won, the third claim). When you change a rule, check both "did the cases that should change change as intended?" and "did the cases that should not change stay the same?" This is also why the IBM ODM page source highlights scenario-based testing and simulation: to do this check before deployment.

The fifth step is new in the LLM era. Testing only the rules is not enough; the extraction prompt and the explanation prompt must be tested together with the same set. If the rule engine stays the same but an edited extraction prompt starts pulling out "subscription period" differently, decisions change silently. Conversely, you might change a rule while the explanation prompt still writes customer notices based on the old criteria. So the test set should include not only structured inputs but also source documents, and run the whole "document → extraction → decision → explanation" pipeline in one go. The same reasoning lies behind the AIP Logic documentation source introducing both Foundry Branching, for developing functions in isolation, and the AIP Evals results analyzer, which groups failed test cases by cause and suggests prompt fixes.

Common mistakes, further study, and on to Part 3

Common mistakes come down to about five. One is writing the decision criteria only in the prompt, with no versioning or tests. Another is the opposite: trying to cover every decision with rules and getting worse results on work the rules cannot handle (recall that Policy-as-Skill paper reported the effect of deterministic controls varies by task). A third is when the decision stage fails to notice errors from the extraction stage. Fourth, the LLM-written explanation does not match the rule the engine actually applied. Fifth, relying only on read-access controls without separately controlling the flow of outputs and edits the model produces source. Validate extraction results against a schema, and always provide an exit that sends empty or ambiguous values to human review, like the last row of the decision table.

For further study, this order is recommended. First, use the Camunda DMN tutorial source to get hands-on with decision tables, FEEL and hit policies, and follow the server policy examples in the OPA official documentation source to get a feel for "separating decision from enforcement." Next, look at DMN-guided prompting paper for the simplest way to connect a standard decision model with an LLM, and the symbolic working memory study paper to understand why LLMs falter when applying rules over multiple steps. Finally, read Policy-as-Skill paper for a design that bundles evidence verification, review routing and auditing into a single runtime, together with its limiting figures.

Once you have decided where to put the rules, the remaining question is "can that decision be trusted?" How certain is the extraction, does the basis for the decision match the actual documents, and can an auditor later trace the decision process exactly? Part 3, "Can the decision be trusted? Confidence, evidence and auditing," covers these three questions.