Why AI in production needs evaluation

A model can look convincing in a demo and still be systematically wrong in production. Only a solid evaluation shows whether it really works.

Managing vending machines throws up a familiar problem again and again: the system reports stock, but on site the slot is empty.

Such phantom stock arises from misfired dispenses, unrecorded removals, reconfigurations or other discrepancies between actual and booked stock. The consequences range from incorrect picking to empty slots that the system continues to treat as filled.

Our solution starts with a simple step:

The service technician photographs the front of the machine with the door open.

A vision model then assesses each slot with exactly one of three outcomes:

• PRODUCT_VISIBLE
• EMPTY
• UNCLEAR

Only afterwards does deterministic application code compare the result with the system stock.

• Model reports EMPTY, system stock is greater than zero: phantom stock
• Model reports PRODUCT_VISIBLE, system stock is zero: unexpected stock

What matters is this: the model does not see the current stock figures. It is meant to describe independently what can be made out in the image, rather than confirm an existing system assumption.

The real problem is the mapping

Whether a slot is empty or filled is usually not the hardest question for a vision model.

Harder is:

Which slot is currently being looked at?

The machine configuration only helps so far. It knows slot labels and logical spans, but no reliable image geometry.

Three configured slots with a span of three each, for example, give nine logical columns. Physically visible, however, there may be six coils and only three actual product positions.

We therefore first tried a geometric approach: the model was to return slot boundaries as coordinates, the code was then to map these regions to the planogram and only then run the stock analysis.

The prototype failed quickly.

The model produced even grids whose numbers looked plausible, but whose boxes did not sit on the actual product positions. Between two runs the estimated shelf boundaries shifted noticeably. Perspective and real mechanics were replaced by an idealised grid.

The key insight from this:

A plausible structure is not yet a correct measurement.

Since then we give the model only the number of actual positions per shelf. If it cannot reliably break a row down into that number, the row is assessed entirely as UNCLEAR.

Uncertainty is a valid result

Every photo is analysed three times with the same prompt.

Only when two runs return the same judgement is it adopted. Contested cases fall back to UNCLEAR and produce no finding.

This helps against random fluctuations, but not against systematic errors. If a model shows the same bias across all three runs, a majority vote can even reinforce that bias.

So it is not enough to look at individual examples.

You need an evaluation.

How the ground truth is built in production

For each photo, the service technician answers three short check questions about selected slots.

These slots are chosen before the model analysis starts. The model therefore cannot influence the selection or the answers.

This creates a continuous, independent reference sample from real production.

In addition, the technician assesses every actual finding:

• confirm
• reject

This produces two kinds of ground truth:

1. spot checks, including for slots without a model finding
2. confirmed or rejected real discrepancies

The dataset grows automatically with every tour. No one has to annotate photos separately for a test project.

The first model comparison

The first production data showed a clear asymmetry.

Phantom stock was already detected robustly. With unexpected stock, on the other hand, the model was often wrong.

The first suspicion was that this direction simply cannot be detected reliably from a front photo.

Shadow mode disproved that assumption.

Other models assessed the same photos considerably better. So the information was there. The problem was not the camera, but the choice of model.

This turned a hunch into a measurable question.

The evaluation: six models, the same photos

Shadow mode gave rise to an internal test bench.

It consists of:

• a fixed, labelled dataset
• identical prompts
• the same three-run procedure
• the same finding logic
• clearly defined metrics
• reproducible campaigns

The first larger run covered:

• 50 audits
• 217 slots with ground truth
• six models
• one cheaper and one more capable variant per provider

The evaluation first confirmed the measurements from shadow mode. That was decisive: two different measurement paths led to the same picture.

Only then was the test bench trustworthy enough to base model decisions on it.

The cheaper model was just as good in the direction that mattered

The most important operational insight was not which model achieved the highest overall accuracy.

It was this:

On phantom stock, the cheap GPT model was just as good as the far more expensive flagship.

Both reached:

• 97 percent precision
• 89 percent recall
• 34 correct findings out of 35 reports

The premium brought no advantage in the economically most important direction.

It only helped with the rarer unexpected stock. This yields a better architecture than “always use the most expensive model”:

• cheap model for regular throughput
• stronger model only as a second instance for the few difficult cases

Evaluation therefore optimises not only quality, but also cost and architecture.

A majority vote does not protect against systematic bias

A particularly telling result came from the small Claude model.

Across 217 assessed slots it almost never reported an empty slot.

The stored explanations showed why: for an actually empty slot, the model repeatedly described the product that, according to the planogram, should be there.

It had turned the expectation from the prompt into an observation.

Individual runs correctly recognised the empty slot. But two hallucinating runs outvoted the correct result.

This shows:

Majority votes help against random instability. Against systematic bias they can cement it.

Without an evaluation, this error would have been hard to spot. The system would have looked stable, because the three runs agreed.

The model does not book anything itself

Even a well-evaluated model must not change stock on its own in this process.

A finding first appears in the back office with:

• photo
• slot
• product
• system stock
• model judgement
• explanation

Only after human confirmation does a regular stock correction arise in the stock ledger.

Only batches that were already in the slot at the time the photo was taken are taken into account. Products restocked later remain untouched.

With unexpected stock, too, nothing is booked in automatically. Without a known batch, shelf life and purchase price, such a booking would be factually wrong.

Evaluation therefore replaces neither business rules nor approvals. It only answers a different question:

How reliable is the model within this controlled process?

The comparison becomes a regression evaluation

The test bench is no longer a one-off model comparison.

Every relevant change gets its own campaign:

• new prompt
• new model
• different image pre-processing
• changed majority procedure
• new business rule

All variants run against the same stored photos and the same metrics.

What would previously have needed several weeks of shadow mode can now be checked within an afternoon.

At the same time, the dataset grows with new, previously unseen images from live operation. This reduces the risk of tuning the prompt ever more tightly to a static test set.

What we take away from this

The most important component of this feature is not the vision model.

It is the evaluation loop:

1. collect independent reference data in production
2. look at types of error separately
3. compare candidate models under identical conditions
4. use stored explanations for error analysis
5. test model changes reproducibly
6. assess quality, cost and risk together

Without this evaluation we would have kept several false assumptions:

• That the front camera fundamentally cannot detect unexpected stock.
• That the larger model is automatically better.
• That three matching runs are a sign of reliability.
• That the configured column structure matches the visible geometry.

None of these assumptions held up to measurement.

For day-to-day operation this means, today:

Stock discrepancies can be detected from a photo that takes about 40 seconds to capture. Every actual correction remains human-confirmed, documented and traceable in the ledger.