When Deciding Gets Cheap

Two Kinds of Hard argued that agents collapsed the cost of producing output while the cost of judgment did not move at all. A new class of model now goes after judgment directly. It does not write anything. It picks one answer from a list you define, says how sure it is, and does that in thirty milliseconds.

The cost of asking fell. The cost of being wrong did not. Most of the new decisions will be made in places nobody is looking.

What these models do, in one paragraph

You hand the model a situation - a support ticket, an email, the current state of an application - along with a question and the list of answers it is allowed to give. It returns one of your answers and a number saying how confident it is. It cannot return anything else, because producing free text is not something it can do. TypeSafe calls its version Jev. The open equivalents are Laya and a smaller model called openJev. All of them answer in twenty to seventy milliseconds, and the cheapest of them run on a laptop.

The prize here is real, and I want to be fair to it before I get to the caveats. When a judgment costs almost nothing and returns in thirty milliseconds, you stop rationing it. You can screen every output instead of sampling one in a hundred. You can route every message instead of deciding once per session. Most significantly, AI agents currently burn expensive tokens deciding what to do next, and moving that decision to a small model makes running an agent roughly ten times cheaper. That is not a marginal improvement. That changes which products are viable.

The cost that fell, and the cost that did not

Asking became nearly free. Being accountable for the answer did not.

Every decision you automate needs labeled examples to tune it, something watching for drift, a path for when it is unsure, and a person whose name is on it. Call that decision upkeep: the recurring cost of keeping an automated judgment honest, which does not fall when the model call gets cheaper. The marginal cost of your fiftieth automated decision is not inference. It is upkeep.

This matters because upkeep arrives late. The savings show up in the quarter you deploy. The upkeep shows up a year later as headcount, and by then nobody connects the two. The business case was written on the half of the ledger that moved first.

The name is a warning

TypeSafe named Jev after William Stanley Jevons, a nineteenth-century economist. His observation was that when something becomes cheaper to use, people use so much more of it that total consumption rises rather than falls. Applied to machine intelligence, that is a growth thesis: cheap decisions mean decisions everywhere.

It is also half the citation. Jevons was writing about coal in 1865, and he was issuing a warning. His argument was that more efficient steam engines would burn through Britain's reserves faster, not slower, because efficiency is not a conservation strategy. Demand expands to swallow it.

Take the warning half seriously and it predicts something specific. Decisions will appear in places that previously had no decision at all - where a default, a fixed rule, or somebody's quick glance was sufficient. Decision volume explodes. The one input that never gets cheaper is somebody checking the answer.

Anyone can build one of these

Jev was announced on 15 September. Laya, an open version with published weights, appeared three days later. Days after that, a third model at roughly 150 million parameters scored slightly higher than both. It was trained in under nine hours on a consumer gaming graphics card and runs in a browser.

Read that as a market signal rather than a leaderboard. The architecture commoditizes on a timescale of weeks. What does not commoditize is labeled examples of your own decisions, because every model in this category needs them. That is the asset. Keep the model layer swappable from the first day, put a thin abstraction over the call, and do not sign anything long.

The scores need reading twice

Laya's headline accuracy on a shared test is 0.766. That figure belongs to a checkpoint fine-tuned on the benchmark's own training split, which is ordinary practice. Without that task-specific training the same model scores 0.362, which its card calls near chance, and its documentation states that fine-tuning is required for production use.

The second number is the one that matters more. Confidence calibration is the gap between how sure a model says it is and how often it turns out to be right. Laya's published calibration error falls from 0.466 to 0.081 - from close to meaningless to genuinely useful - but only after refitting a temperature on labeled examples from the specific task in front of it.

Credit where it belongs: Laya published both numbers itself. Jev publishes neither. It also publishes no parameter count, no named benchmark, and no calibration figure. Calibration is asserted rather than evidenced. That asymmetry is worth more attention than the benchmark race.

One related claim deserves care, because it is the most misread thing in this category. Zero hallucination means the model cannot return a value outside the list you gave it. It does not mean the answer is correct. A well-formed answer and a right answer are different things, and the first guarantee tells you nothing about the second.

You will use it most where it works least

Here is the tension, and it is not resolved by better benchmarks.

Confidence is a property of the examples a model was tuned on, not a property of the decision sitting in front of it. Cheap decisions spread into the unusual, one-off situations that were never worth a model call before - rare cases, new customers, odd edge cases. Nobody has labeled examples of those. So the economics push these models precisely into the territory where their confidence means least, and a well-tuned model does not know when it has left familiar ground. It reports confidence in the same voice either way.

The failure this produces is not a visibly wrong answer. It is the quiet wrong: a decision that is mistaken, confidently made, and invisible because nothing broke. The router sent a customer down the adequate path instead of the good one. Quality degrades in a way your incident process cannot see, because there is no incident. Nobody files a bug for a slightly worse experience.

At ION Mobility, a dashboard prototype showing stale data became the source of truth for investor reporting. Nobody decided to promote it. It crept because it was useful, and nobody asked whether its governance had kept pace with its importance. A decision model makes that creep faster and quieter still, because it can be wired in over an afternoon by whoever needed it and then left unowned. Ownership clarity is the control that fails first.

The honest position is uncomfortable. You are trading throughput you can measure now against reliability you cannot measure at all, and that trade is genuinely difficult to defend to a board. Anyone selling this as a free productivity gain is describing the first half of Jevons and leaving before the second.

Knowing when to decline is the feature nobody benchmarks

Two of the three models ship a way to decline - to hand the decision back to a person or a larger model when the input is unfamiliar. Jev discloses no such mechanism. And none of the three publishes a score for how reliably it declines on inputs it has not seen.

That is the number that bounds blast radius. Not accuracy on a familiar test. How often the model correctly says it is not sure, on the long tail where it will actually spend most of its life. It is the hardest number to produce and the only one that tells you what happens when the system is wrong at scale.

That points at design discipline rather than model choice. At a hundred times the volume you cannot review the output, so recovery design replaces accuracy as the thing you engineer for. Suggestions rather than actions. Soft routing rather than hard routing. Staged rollout rather than a switch. Make a wrong decision cheap to reverse instead of hoping it is rare.

Two questions for your engineering team

Which automated decision in our product has no owner? Find the one somebody built in an afternoon because they needed it. Check whether anyone has looked at its outputs since. If the answer is no, you do not have a model problem, you have an unclosed feedback loop, and the model is making it faster.

What happens when it is unsure? If the answer is that it returns its best guess anyway, then you have not deployed a decision model. You have deployed a confident one, and the difference will only become visible in aggregate, months from now, in a metric nobody is currently watching.