Trust Is Not a Score: A Mathematical Framework for Quantitative AI Assurance

A transparent mathematical framework for AI assurance should preserve scope, evidence strength, coverage, uncertainty, critical gates, and accountable human decisions instead of compressing trust into one number.

The desire to express the trustworthiness of an artificial intelligence system as a score is understandable. A number appears comparable, trackable, and suitable for a dashboard. It can also conceal the information a decision-maker most needs.

An AI system may have excellent predictive performance and inadequate recourse. It may have strong documentation and a failed authorization control. It may pass every evaluated test while half of the applicable control set remains unevaluated. Averaging these conditions into a single number creates precision without necessarily creating knowledge.

My proposition is simple: quantitative AI assurance should be mathematical, but trust should not be reduced to a score.

Assurance is a scoped claim

Every assurance result should begin with a scope:

A measurement for one scope is not automatically valid for another. A material change in any component of $S$ may require a new result. This is especially important for AI systems because performance and risk can change with data, prompts, retrieval sources, connected tools, providers, users, and operating conditions.

The scope should therefore be part of the result, not a footnote.

Separate the dimensions

Let $D$ be the set of assurance dimensions relevant to the scope. Depending on the system, these may include:

• purpose, impact, and affected parties;
• data and knowledge quality;
• model and provider governance;
• prompt, retrieval, and tool authority;
• evaluation quality and robustness;
• security, privacy, and provenance;
• release and deployment integrity; and
• operations, incident response, and recourse.

For each dimension $d$, let $C(d)$ be the set of applicable controls. Applicability is a governed decision, not a numerical convenience. If a control is not applicable, the rationale should be recorded and independently reviewable. Not applicable is not the same as zero, missing, or passed.

An evidence-adjusted dimension value

Let $E(d)$ be the evaluated controls in dimension $d$. For each evaluated control $i$, define:

• $w(i)>0$: its approved importance weight;
• $q(i)\in [0,1]$: the normalized observed control result;
• $e(i)\in [0,1]$: the strength of the supporting evidence; and
• $f(i)\in [0,1]$: its freshness factor.

The evidence-adjusted value for the evaluated portion of a dimension is:

This expression has a limited purpose. It summarizes the evaluated controls in one dimension. It is not, by itself, a deployment decision or a claim of trustworthiness.

Domain assumptions and boundary cases

The equation is defined for a finite, nonempty evaluated set $E(d)$ contained within the applicable set $C(d)$. All weights are strictly positive, so the denominator is then positive. If no applicable control has been evaluated, the assurance value is undefined and should be reported as Not assessed, not as zero.

Under these assumptions, the value is bounded:

Holding the scope, evaluated set, weights, and all other inputs fixed, increasing any one of the observed result, evidence-strength, or freshness factors cannot decrease the value. This monotonicity is useful, but it does not make the result a probability, a calibrated confidence level, or a measure that can be compared across scopes with different rubrics and weighting policies.

An evaluated control should have admissible evidence. Unsupported controls remain outside the evaluated set rather than receiving an arbitrary evidence factor of zero. This preserves the distinction between a demonstrated control failure, weak but admissible evidence, and no assessment.

The observed value $q(i)$ can be derived from an approved control rubric. A useful ordinal rubric distinguishes five levels:

0. Missing: no adequate implementation evidence.
1. Defined: the control and owner are documented.
2. Implemented: the control operates in the stated scope.
3. Verified effective: current evidence independently demonstrates effectiveness.
4. Continuously evidenced: governed monitoring repeatedly demonstrates effectiveness.

If the rubric is normalized, one possible mapping is $q=\frac{\text{level}}{4}$. That mapping should not imply that the conceptual distance between adjacent ordinal levels is scientifically equal. Where the distinction matters, report the level itself rather than claiming interval-scale precision.

Coverage must remain visible

An assurance value can increase when difficult controls are left unevaluated. Coverage is therefore a separate, mandatory quantity:

If every applicable control has equal weight, this reduces to evaluated controls divided by applicable controls. A result such as $A(d)=0.86$ and $K(d)=0.42$ should not be summarized as “86 percent assured.” It means the evaluated 42 percent of applicable control weight produced an evidence-adjusted value of 0.86.

Organizations should establish a minimum coverage threshold before a dimension can support a positive decision. Mandatory controls should never disappear merely because the weighted coverage threshold was met.

The coverage equation assumes a finite, nonempty applicable set. If no controls are applicable, the dimension should be marked Not applicable with a reviewable rationale; it should not be assigned perfect coverage. If applicable controls exist but none has been evaluated, coverage is zero and the assurance value remains undefined.

Critical gates precede aggregation

Let $G$ be the set of critical gates, and let $g(j)\in \{\text{pass},\text{fail},\text{unknown},\text{stale}\}$ describe gate $j$. Let $O$ indicate whether an active operational stop condition exists.

A defensible decision function applies precedence:

This is a non-compensatory model. No weighted average can override an earlier blocker. The order matters because some conditions represent boundaries rather than preferences.

These labels are governance states, not statistical conclusions. The stated precedence is a policy choice: when several conditions coexist, the first applicable state is reported. An organization may adopt different labels or precedence, but it should version the policy, apply it consistently, and preserve the underlying conditions so reviewers can see more than the headline state.

Uncertainty belongs beside the result

Many AI evaluations estimate a rate from a finite sample. Suppose $x$ qualifying outcomes, defined in advance as successes, are observed in $n$ trials, where $n>0$ and $0\le x\le n$. Reporting only $\frac{x}{n}$ hides sampling uncertainty.

For a binary outcome measured through conditionally independent and exchangeable Bernoulli trials with a stable success probability over the stated scope and period, one transparent Bayesian model begins with a beta prior whose parameters $\alpha ,\beta >0$:

The posterior mean is:

The organization should publish the prior, the reason for choosing it, the sampling protocol, exclusions, and a credible interval rather than presenting the posterior mean alone. If trials are dependent, the population changes during measurement, or the outcome is not binary, this beta-binomial example is not justified without an alternative model. A frequentist confidence interval may be preferable in another governance context. The important point is not to mandate one statistical school; it is to make the uncertainty model and its assumptions reviewable.

Sampling uncertainty is only one type of uncertainty. Dataset shift, measurement error, evaluator disagreement, incomplete coverage, and model or provider changes should not be folded into the same interval without justification.

Evidence strength and freshness are not decoration

The evidence factor $e(i)$ should reflect an approved hierarchy. Independently reproduced operational evidence may carry more weight than an unverified assertion. A policy may strongly support that a control was defined while providing little evidence that it worked.

A simple freshness model might be:

However, linear decay is not universally appropriate. Some evidence remains valid until a material change; some expires abruptly; some requires continuous monitoring. The freshness function should match the evidence contract and should be visible to the reviewer.

For the illustrative linear function, age is nonnegative and the evidence horizon must be positive. A material change may force freshness to zero immediately, irrespective of elapsed time. Evidence-strength and freshness rules should also be checked for double discounting—for example, an evidence grade should not already penalize age if the freshness factor applies the same penalty again.

Residual risk is not the inverse of assurance

Assurance and risk answer different questions. Assurance describes the supported confidence that defined controls or criteria operate as claimed. Risk concerns uncertain events and their consequences.

For scenario $k$, a simple expected-loss representation is:

Here $P$ is estimated pre-mitigation likelihood between zero and one, $I$ is nonnegative impact in a stated unit, and $M$ is a proportional mitigation effect between zero and one. If likelihood or impact is already estimated after mitigation, multiplying by the mitigation term again would double-count the control effect. Each term can be deeply uncertain, dependent, and difficult to place on a common scale. The equation is a schematic for exposing assumptions, not a generally calibrated risk model; it is harmful when it turns weak judgments into an apparently exact monetary value.

Scenario risks should not be summed without considering common causes, dependencies, mutually exclusive events, and impact units. Where those conditions cannot be modeled credibly, ordinal risk bands with explicit rationales may be more honest than a numerical total.

A high assurance value does not imply low residual risk. A well-controlled system can retain high inherent consequence, and a low-risk use can have immature controls.

Test the model for sensitivity and gaming

Weights and thresholds encode judgment. Their influence should be tested.

For a continuously varying parameter $\theta$, local sensitivity of dimension $d$ can be expressed as:

The derivative applies only where the assurance expression is differentiable. Gate thresholds and the final status are discrete and can change discontinuously, so finite perturbations and scenario analysis are usually more informative: vary important weights, evidence factors, thresholds, and missing-data assumptions across approved ranges and observe whether the value or decision changes.

A metric also needs an explicit gaming analysis. Ask what behavior would improve the number without improving the underlying control or outcome. If a team can raise the score by narrowing the denominator, selecting an easy sample, delaying incident classification, or attaching low-quality evidence, the measurement contract is incomplete.

A worked example

Consider a dimension with four applicable controls of equal weight. Three are evaluated:

The evaluated dimension value, rounded to three decimal places, is:

Coverage is:

The honest statement is not “the system is 65.8 percent trustworthy.” It is: “For this scope, the evaluated portion of this dimension has an evidence-adjusted value of 0.658, with 75 percent control coverage. Recourse validation remains unevaluated.” If recourse is mandatory, the decision is Not assessed regardless of the aggregate.

Institutional safeguards around the mathematics

The formula alone cannot make assurance trustworthy. The operating process should require:

• versioned definitions for scope, population, formula, thresholds, gates, and limitations;
• independent approval where the preparer has a conflict or could benefit from the result;
• immutable snapshots tying results to exact source and definition versions;
• distinct states for missing, zero, unavailable, stale, failed, and not applicable;
• recorded material-change and reassessment rules;
• separation between control assurance, operational outcomes, and residual risk; and
• controlled reporting that preserves lineage, classification, redaction, and access history.

Relationship to established assurance frameworks

This proposal is a measurement architecture, not a replacement for a governance, risk-management, conformity-assessment, or audit standard. It is intended to sit inside an organization’s approved governance and evaluation practice.

The versioned references used here include the NIST Artificial Intelligence Risk Management Framework 1.0, which organizes work around governing, mapping, measuring, and managing AI risk, and the NIST Generative Artificial Intelligence Profile. ISO/IEC 42001:2023 specifies requirements for an organizational AI management system and continual improvement. The UK Department for Science, Innovation and Technology describes assurance as measuring, evaluating, and communicating whether systems meet relevant criteria in its Introduction to AI assurance and illustrates multiple techniques in its Portfolio of AI assurance techniques.

Within the European Union, Regulation (EU) 2024/1689 establishes rules for AI systems and general-purpose AI models, including requirements that vary by role, system classification, and applicability date. This proposed framework neither determines whether the Regulation applies nor demonstrates compliance. Organizations should consult the current consolidated legal text and obtain qualified legal advice for their circumstances.

Limitations and validation agenda

The model has not yet been calibrated against incident outcomes, inter-rater agreement, audit findings, or external assurance decisions. Its factors may be correlated; control weights and evidence grades remain judgment-dependent; multiplication can create stronger penalties than decision-makers intend; and different dimensions may require different measurement scales. Consequently, the numerical values should support structured review, not substitute for it.

A validation program should test at least four questions: whether independent reviewers assign comparable control results and evidence grades; whether results are stable under reasonable changes in weights and thresholds; whether critical gates identify conditions that historical aggregate scores would have concealed; and whether the framework predicts or explains outcomes that matter without encouraging metric gaming. Until such studies are completed, this proposal should be described as a conceptual framework rather than a validated assurance instrument.

Conclusion

Quantitative assurance is valuable when it disciplines judgment, exposes assumptions, and makes evidence easier to challenge. It becomes dangerous when it compresses scope, missing information, critical failures, and uncertainty into an attractive number.

The goal is not a universal trust score. The goal is a defensible assurance statement:

For this exact scope and period, under these approved definitions, with this evidence, coverage, and uncertainty, the critical gates produced this decision—subject to these limitations and reassessment triggers.

That statement is longer than a score. It is also far more useful.

References and source versions

1. Tabassi, E. (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1. National Institute of Standards and Technology.
2. Autio, C., Schwartz, R., Dunietz, J., Jain, S., Stanley, M., Tabassi, E., Hall, P., and Roberts, K. (2024). Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile. NIST AI 600-1. National Institute of Standards and Technology.
3. International Organization for Standardization. (2023). ISO/IEC 42001:2023 Information technology—Artificial intelligence—Management system.
4. UK Department for Science, Innovation and Technology. (2024). Introduction to AI assurance.
5. UK Department for Science, Innovation and Technology. (2023). Portfolio of AI assurance techniques.
6. European Parliament and Council of the European Union. (2024). Regulation (EU) 2024/1689 (Artificial Intelligence Act). Official Journal of the European Union.

Dr. Ashwini Kumar Rath is an entrepreneur, author, and researcher.