Working AtlasAI evaluation & review

Confidence Review Bench

Move a detection threshold across known examples and inspect the alerts and misses it creates.

TryAt Overlap 65%, lower the threshold to 50%; compare false alerts and missed appearances.

Watch forIn the overlap sets, lower the threshold: more true appearances are found, but more absent frames become false alerts.

Jump to the model

Separate teaching model

The linked story describes related work; the scenarios and results here are synthetic.

Interactive model

One line, two kinds of error

2 absent frames still trigger an alert, while 2 real appearances fall below the threshold. Inspect those frames before choosing a policy.

Set the decision line

Synthetic frame set

Some target and background scores sit near the boundary.

Detection threshold65%

Scores at or above 65% are marked as detections. The labeled examples stay fixed while you move the threshold.

No target

Clear

21%

Precision

67%

Correct detections ÷ all detections

Recall

67%

Found appearances ÷ known appearances

False alerts

Absent frames marked as detections

Missed appearances

Target frames below the threshold

Fixed teaching examples only. Scores are illustrative and are not predictions, measured product data, or calibrated model confidence.

Preparing the interactive experiment…

Skill practicedEvaluate confidence thresholds against labeled examples

Connection to my workNeuralEye documents confidence scores and selective verification; this synthetic set explores threshold trade-offs, not its calibration.

Related project:NeuralEye project story →

Understand the result

Each fixed, synthetic video-frame set has a known label and an illustrative confidence score. The threshold turns a score into a detection; the bench compares those decisions with the labels and updates precision, recall, false alerts, and missed appearances. No model is running and the scores are not calibrated production data.

Controls

Synthetic frame set: Choose examples with clean separation, some score overlap, or heavy overlap between frames that contain the target and frames that do not.
Detection threshold: Frames at or above this score are marked as detections. Move the threshold to change the decision line, not the underlying examples.

What to watch for

In the overlap sets, lower the threshold: more true appearances are found, but more absent frames become false alerts.
Raise the threshold and inspect the frames that become misses; higher confidence is not automatically better coverage.
Compare precision and recall alongside the frame list. Neither metric alone tells you the cost of a false alert or a missed appearance.

Why it happens

A threshold converts a continuous score into a yes-or-no decision. Moving it changes which labeled examples are counted as detections. Precision describes how many marked detections are correct; recall describes how many known appearances were found. The three fixed sets are teaching fixtures, not representative samples, calibrated scores, or a quality claim about any production model.

How this idea relates to the linked work

NeuralEye’s documented detection flow includes confidence scores, layered proposals, and selective verification. This independent synthetic bench helps me reason about threshold trade-offs against known labels; it does not use NeuralEye scores or reproduce its calibration or review policy.

See the broader engineering brief