Research paper

A Confidence Estimation Framework for Skin & Product Audits

Qutis AI Research Team·June 18, 2026·Published

AbstractAutomated audits must know when they are uncertain. This paper describes how Qutis converts model token log-probabilities into per-observation confidence scores, aggregates them across multiple audit concerns, and routes low-confidence cases to qualified human review — with referral to a licensed medical professional when warranted.

System at a glance

The audit pipeline collects context through user interaction, runs a pattern-recognition model over images and text, applies an ingredient-safety rules engine, and scores confidence before any report ships.

Confidence alone is not proof of correctness. It is interpreted alongside prior validation on an independent held-out evaluation before scores are trusted for routing decisions.

Horizontal flow from context intake through pattern recognition, ingredient rules, confidence scoring, escalation gate, to audit report. — Conceptual audit pipeline. No single step ships a report without passing the confidence gate or human review.

Structured observations from the model

Given a photo and collected context, the pattern-recognition model produces structured visual observations — a primary pattern label and secondary notes calibrated for the user's Fitzpatrick type.

We use neutral example labels in documentation (e.g. "pattern A") rather than named conditions. The same confidence machinery applies regardless of the observation text.

Input photo flows through a model box to a structured output with neutral observation fields. — Input → model → structured observations. Target tokens are parsed without diluting the signal with descriptive filler.

From logits to probabilities

Each generated token is selected from a probability distribution over the vocabulary. Raw logits are normalized with softmax so probabilities sum to one.

p_i = \frac{e^{z_i}}{\sum_j e^{z_j}}

Softmax over logits zᵢ. Here pᵢ is the probability of token i and j indexes the full vocabulary.

Bar chart of softmax probability mass across vocabulary tokens. — Probability mass concentrates on a few tokens; the rest of the vocabulary shares the remainder.

Working in log-space

Our serving stack returns log-probabilities for each generated token rather than raw softmax values. Log-space is additive, which simplifies aggregation across multi-token observations.

Individual log-probabilities are recovered to standard probability space by exponentiation.

\ell_i = \log(p_i) \quad\Rightarrow\quad p_i = e^{\ell_i}

Log-probability ℓᵢ and its inverse via the natural exponential.

Diagram mapping log-probability space to probability space via exponentiation. — Exponentiation reverses the log transform and restores a value in (0, 1].

Scoring a single observation

For a target span of m tokens with log-probabilities ℓ₁…ℓₘ, we compute the arithmetic mean of log-probabilities, then exponentiate. Because log-probabilities are additive, this is equivalent to the geometric mean of the underlying token probabilities.

\bar{\ell} = \frac{1}{m}\sum_{k=1}^{m}\ell_k

Mean log-probability across the m tokens in a single observation span.

Geometric-mean equivalence

The confidence score C for one observation is the exponential of the mean log-probability — numerically identical to the geometric mean of per-token probabilities.

C = e^{\bar{\ell}} = \left(\prod_{k=1}^{m} p_k\right)^{1/m}

Confidence C as geometric mean of token probabilities.

Per-token probabilities collapsing into one confidence score via geometric mean. — Multiple token probabilities collapse into a single confidence value C.

Aggregating multiple observations

A single audit may yield several related observations — a primary pattern plus secondary notes. The extraction procedure runs independently for each concern, producing C₁, C₂, …, Cₙ.

The final case confidence is the arithmetic mean of the individual observation scores.

C_{\text{final}} = \frac{1}{n}\sum_{i=1}^{n} C_i

Aggregated confidence across n distinct observations in one audit.

The escalation gate

Before each audit, a certainty threshold τ is set. If the computed confidence falls below τ, the request is immediately routed to qualified human review with all collected context attached.

When human review surfaces a case that may need in-person care, we refer the user to a licensed medical professional. Reports that pass the gate ship back through the same messaging channel.

Confidence compared to threshold: above ships the audit, below triggers human review and possible medical referral. — The escalation gate is the safety boundary between automated output and human oversight.

References

Vaswani, A., et al. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30.

QUTIS AI IS A SKINCARE AUDIT AND ADVISORY SERVICE. IT IS NOT A MEDICAL ASSESSMENT TOOL AND DOES NOT REPLACE PROFESSIONAL MEDICAL CONSULTATION.

Join the waitlist