Most engineering teams do not have a model problem — they have a decision problem. Whether a support ticket is urgent, whether an order should ship, whether a frame from a production line shows a defect: these are decisions that have to be made millions of times with a stable contract and a known error budget. A decision API makes that transaction explicit. You send state, you ask questions defined in code, and you get back a typed answer with a calibrated confidence value attached. No prose to parse, no regexes bolted onto a response body, no heuristics scattered across five services.
That distinction matters because software that consumes free-form generated text is unreliable at scale. A JSON-shaped chat prompt is a hope, not a contract: the model can add a preamble, rename a field, or return “likely yes” where your parser expects true. A decision API flips the responsibility. The schema lives in your codebase, the answer arrives already typed, and every response carries a number that tells you how much to trust it.
What a Decision API Is — and How It Differs From an LLM in a JSON Wrapper
The call shape is deliberately boring. You POST to a single HTTPS endpoint with two things: state and questions. State can be plain text, structured JSON, or up to 16 images or video frames. Questions are declared in code, which means the response schema is something you control and version like any other interface in your system. The engine evaluates everything in one pass and returns answers that are already schema-valid.
The answer types are the interesting part, because they map directly onto the primitives engineers already reason about:
- choice — exactly one label from a set you defined (for example billing, shipping, account-access)
- score — a level on an ordered scale, useful for severity or priority tiers
- truth — a calibrated probability between 0.00 and 1.00, not a vibe
- number — a value snapped to a grid you specify
- point and box — locations inside an image, for spatial questions
Contrast that with the “reply only in JSON” pattern that teams reach for first. Every request pays for autoregressive decoding of tokens the caller will immediately throw away. Prompt drift means the schema you validated in staging is not the schema you get in production three weeks later. And because generation is sampled, the same input can produce a different answer on the next retry, which makes incident replay nearly impossible.
A purpose-built decision API avoids all three. The output type is fixed per question, so validation is not a post-processing step — it is the boundary itself. Responses are deterministic: the same request returns the same answer, which means you can write golden-file tests, replay a production request during a postmortem, and get the identical result. For teams replacing brittle hand-written rule trees, that determinism is often the deciding factor: you get the flexibility of a model with the reproducibility of a function.
There is also an operational argument. Rules engines accumulate exceptions until nobody can explain why a specific account was flagged in March. A decision API collapses that logic into a declared question with an observable answer and a confidence value, so the audit trail is the request itself.
Calibrated Confidence Is the Real Contract
The most useful thing a decision API returns is not the label — it is the number next to it. An answer without confidence forces your code into a binary: trust it fully or ignore it. An answer with a calibrated probability lets you build a three-lane policy: act instantly above a high threshold, take a safe default path below a low threshold, and route the ambiguous middle to a human or a slower, more expensive check.
Calibration is the word doing the work here. A score of 0.90 should mean that across all the cases where the engine said 0.90, roughly nine in ten were correct. That property is what makes thresholds portable between environments and defensible in a review. If your auto-approval gate sits at 0.95 for promo-abuse checks, you can state an expected false-positive rate rather than a feeling. This is the difference between a system you can put an SLA behind and one you monitor by watching dashboards nervously.
The practical pattern looks like this in almost every production integration. A request arrives with state and a small set of questions: is this message a refund request? (choice), how urgent is it? (score), does this image show damage? (truth). The response returns typed answers, each with its own confidence. Your code branches on the numbers, not on parsing. Above threshold, the workflow advances without a human in the loop. Below it, the case is re-checked with richer state — more images, additional context, a second question set — or handed to an operator with the confidence displayed so they know how much scrutiny to apply.
This is the architecture engines such as decisionnode.com are built around: every answer carries a calibrated confidence, so the calling service can act immediately when the number is high and take a safe path when it is not. Because the returned structure is typed rather than generated as text, there is no schema drift to reconcile and no token-level cost for making the response explicit. The caller decides what “confident enough” means; the API’s job is to report the probability honestly.
One design note: pick your answer type deliberately. If your downstream logic is a branch, ask for a choice. If it is a queue priority, ask for a score. If you intend to threshold it, ask for truth and get the probability directly rather than reverse-engineering one from a label.
Latency, Determinism, and Throughput in Production
A decision API only earns its place if it fits inside the latency budget of the thing calling it. Interactive workflows — a support console, a checkout, an agent loop — break down when a single hop takes seconds. DecisionNode publishes two model tiers for this reason. DecisionNode Flash runs at roughly 5 ms server-side and about 19 ms end to end, priced at $0.035 per 1M input tokens. DecisionNode MoE runs at roughly 9 ms, priced at $0.042 per 1M input tokens. Output tokens are always free, which changes the economics of asking many questions per request: a richer response costs nothing extra beyond the input state you send.
Throughput is handled in two modes. For request/response traffic, the HTTPS endpoint plus batch discounts covers the high-volume, bursty patterns typical of triage queues and moderation pipelines. For tight loops — drones, robots, simulators, machine monitoring — where a decision is needed more than ten times per second, sessions over WebSocket remove the per-request connection overhead and keep the loop stable. There is a free daily tier, which is genuinely useful for building and testing a schema before committing to a rollout.
Determinism deserves emphasis on the ops side. Because identical requests return identical answers, a decision API behaves like a pure function from the caller’s perspective. That means you can run shadow traffic against a candidate question set and compare decisions case by case. It means a flaky integration test is a real bug in your code, not model sampling noise. It also means cost modelling is straightforward: input state scales linearly, output structure does not.
Integration surface matters too. Client SDKs for TypeScript, Python, Go, and Ruby, plus plain curl, mean the first call is minutes away rather than a sprint away. Webhooks let decision results fan out into existing queues and alerting, and MCP tooling lets AI agents call the decision endpoint before acting — a pattern that turns an agent’s improvisation into a checked precondition. Treat the question schema like any other API contract: version it, snapshot it, and include the confidence in your logs so you can retroactively tune thresholds against real traffic.
Where Decision APIs Land in Real Systems
The workloads that benefit most share a shape: high volume, a small number of well-defined questions, and a cost attached to being wrong. Ticket routing and triage is the canonical example — a single call can classify the issue, estimate severity, and flag whether the message contains a cancellation request, replacing a rule tree that grows every sprint. Fraud and promo-abuse checks fit the same pattern, with the calibrated probability driving an approval, a block, or a manual review queue. Order and shipping decisions, moderation, risk scoring, and feature gating all reduce to the same primitive: state in, typed judgement out, confidence attached.
Physical and visual contexts are where the multi-modal options earn their keep. Sending up to 16 images or video frames in one request lets quality-control checks on production lines evaluate a sequence rather than a single snapshot, and lets warehouse or traffic monitoring systems ask spatial questions directly — a box for the pallet, a point for the obstruction. Boosters extend this without extra plumbing: OCR for labels and paperwork, face detection and recognition for access flows, human age estimation for restricted-content gating, AI-generated image detection for marketplace and KYC abuse, and depth or segmentation for scene understanding. Each returns the same typed answer contract as everything else.
For AI agent builders, the value is even more direct. Agents are good at planning and bad at knowing when they are about to do something irreversible. Inserting a decision call before the action — at 19 ms end to end — gives the agent a checked precondition with a confidence score, so the orchestration layer can proceed, escalate, or stop. The agent stops guessing and starts asking a system designed to answer.
Migration paths tend to look similar across teams. Start with one high-friction rule set that has become unmaintainable — refund eligibility, say, or shipping exceptions. Express it as questions in code, run it in shadow mode against production traffic, and compare. When the typed answers line up with the outcomes you already trust, move the threshold from observe to enforce. The rule tree does not get deleted; it collapses into a schema, a confidence threshold, and a fallback path that someone can actually explain to an auditor.
Casablanca data-journalist embedded in Toronto’s fintech corridor. Leyla deciphers open-banking APIs, Moroccan Andalusian music, and snow-cycling techniques. She DJ-streams gnawa-meets-synthwave sets after deadline sprints.
Leave a Reply