Why be uncertain?

Everyone loves certainty. Why wouldn’t you? Certainty is clean; it gives us that sweet comfort of a world in which the next thing follows predictably from the last. Reality, however, is nothing if not uncertain.

Last week, TypeSafe released Jev, a somewhat strange little model that performs a simple task. You give it some unstructured state and a contract specifying the decisions you want it to make, and Jev gives you decisions and probabilities. The training method TypeSafe uses is Reinforcement Learning for Calibrated Decisions, or, more recursively, rewarding the model for being right about how often it’s right. The result is that Jev is pretty good at living up to the probabilities it assigns¹.

Though I happen to think this is quite profound, there is, as always, a cohort of the unimpressed. After all, LLMs, SLMs, classifiers, logistic regressions, and… well… even elementary linear algebra have been emitting decisions that satisfy schemas for a long time. And this is true — decisions, as it were, are cheap.

One of the most basic pieces of machinery imaginable, a coin flip, is already a surprisingly powerful decision maker. Randomness is a bona fide computational resource (randomised algorithms can often do things more efficiently than their deterministic counterparts), and randomness provides the kind of variation necessary for exploration, which biology has been exploiting far longer than we have².

None of this says anything about making good decisions. The thing that makes Jev interesting is not that it can make a decision, but that it can do so over a wide variety of tasks while preserving the notion of how uncertain it is about that decision. That turns out to buy quite a lot.

So, why is uncertainty so valuable?

First, uncertainty makes decisions composable — or at least makes their composition well-defined. Imagine three pieces of software arranged in a pipeline that evaluate, say, whether a transaction looks legitimate, whether the account looks trustworthy, and whether the requested action is consistent with the account’s normal behaviour. If each of these decisions outputs just a YES/NO answer, we know very little about the reliability of the final decision. Suppose, on the other hand, that they have confidences of 0.95, 0.90 and 0.85 respectively. We now retain something about the epistemic state of each decision, and if these are the relevant conditional probabilities and the correctness events are conditionally independent, the final outcome can be assigned a probability of \(0.95 \cdot 0.90 \cdot 0.85 = 0.727\). In essence, if you keep decision-level probabilities, then you get a case-by-case downstream probability rather than knowing just that, on average, the system is correct 85% of the time.

Second, uncertainty lets us be intelligently random. This is particularly useful whenever making decisions is really a form of search. Always choosing whatever currently looks best means you are repeatedly searching the most promising part of the space. That works well unless you’ve found a local maximum, in which case a little bit of stochasticity goes a long way³. Suppose Jev is helping us decide which of 10 hypotheses deserves an experiment. If we always choose whichever hypothesis currently has the highest probability, then we’d be repeatedly exploiting what currently looks best. A better option would be to sample in proportion to our beliefs, so that a hypothesis assigned 60% of the probability mass gets explored much more often than one carrying 5%, but the 5% hypothesis still occasionally does get explored.

Third, and most practically, uncertainty is a great way to tell when to “try harder”. We’ve been using Jev to triage the amount of reasoning power we throw at different problems. Rather than spending the same amount of compute on each instance, uncertainty gives us a fairly simple way to spend intelligence where it is actually needed (i.e., when Jev is uncertain).

Adopting under uncertainty

And on a self-reflective note, you may have noticed that software is moving rather quickly. Jev had existed publicly for roughly two days when we started putting it into our systems at Valency. There is obviously some irony here. A post extolling calibrated uncertainty is being written by people enthusiastically adopting a model whose long-run usefulness is itself highly uncertain! But that’s also the point. We are entering into an increasingly non-deterministic world in which you need to be comfortable with a little uncertainty to keep up. The trick, again, is to be principled about it! Being comfortable with uncertainty doesn’t just entail blind faith, it means testing things rigorously in your own systems, figuring out what works, and updating accordingly. At Valency, we’re excited about finding new and unexpected ways to use these new tools. If this kind of work sounds exciting to you, come build with us.

References & Further Reading