Your review queue is a budget

Every threshold makes a promise that nobody wrote down. Draw the review line a little lower and more cases go to people. Draw it a little higher and fewer do. The line looks like a technical setting. It is really a staffing decision.

Most teams set the line first, from a tutorial default or a gut feeling, and discover the workload later, when the queue is full and reviewers start skimming. The order should be the other way round: start from what your team can actually review, and draw the lines to fit.

Count the budget first

The budget is simple arithmetic. Take an example: three people who can each give six hours a day to reviews, at about ten cases an hour. That is 180 careful reviews a day. Not 180 glances: 180 decisions a person can stand behind.

Whatever your own numbers are, write them down. Include the honest parts: meetings, breaks, the cases that take an hour instead of six minutes, the days someone is away. A budget that assumes perfect days will be overspent every day.

The review band is not a setting. It is a budget, and it is measured in people's hours.

Spend it where people change the outcome

Three doors share the work. Act on the cases the model is confident about and that are cheap to get wrong. Escalate the ones that are both likely and costly. Review the middle, where a person's judgement actually changes what happens.

With a budget in hand, drawing the lines becomes a filling exercise. Sort cases by how much a review is worth: how uncertain the model is, and how much a mistake would cost. Fill the review band from the top of that list until the budget runs out. Everything above the band is escalated; everything below it, the machine closes.

This only works if the scores are honest. If the model says 0.2 and is really right four times in ten at that score, the cases you are closing are riskier than you think. Check calibration from your logged outcomes before you trust any line.

When volume grows, something has to give

Cheap decisions invite more decisions. When volume doubles, the review budget doesn't. There are only two honest answers: narrow the band, so people review only the cases that matter most, or grow the team. A line set as a share of volume does neither. It sends more cases to the same people every month, and nobody notices until the reviews stop being reviews.

That is the Jevons paradox of decisions from an earlier letter, landing on the people in the middle. The fix is the same: set the lines by cost and capacity, not by share.

Watch the queue

A budget needs a check. Once a week, look at three things in your decision log: how full the review queue was, how often reviewers overruled the model, and how long cases waited. A queue that is always full means the lines were drawn by hope. A queue that is often empty means you may be paying people to approve what the machine could close.

None of this is complicated. It is a number of hours, a sorted list and a weekly look. But it turns a threshold from a guess into a promise you can keep, to your customers and to the people doing the reviewing.