Skip to the evidence
AI Trading SignalsDefinition / Population / Proof

AI claims / Consumer evidence

Missing Outcomes and AI Accuracy Denominators

Keep unresolved, expired and selectively retained predictions visible beside any quoted accuracy or positive fraction.

Define the eligible prediction population

A reported percentage needs a stable population. Identify whether the claim covers all model outputs, emitted alerts, actionable instructions, closed hypothetical decisions or actual filled account trades. Those sets need not be identical. Preserve IDs, versions, timestamps and the chosen endpoint convention. The scoring reference defines metrics on supplied observations; it cannot establish that a commercial dataset includes every observation its advertisement claims to cover.

A missing outcome is not automatically positive, negative or harmless. Retain why it is missing and whether the observation was eligible under the frozen method. An open horizon, absent data, expired instruction and unfilled order are different states. Do not delete them because they complicate a clean figure. A complete publication history can still coexist with incomplete account execution evidence; report both populations rather than silently substituting one for the other.

Closed-only and all-issued questions differ

A closed-only fraction can describe its declared subset honestly. It becomes misleading when the heading implies that every issued output resolved under the same convention. Counts for unresolved observations help a reader understand coverage, but they do not justify guessing their future outcomes. A model-quality review should preserve the original cutoff and avoid choosing a later endpoint merely because the unresolved cases eventually look favourable.

Do not multiply update messages into extra wins

A partial target, an amendment or a repeated delivery may belong to one original idea. Keep its event history rather than expanding the number of independent predictions after a successful move. If the evaluated unit is an account quantity, weight actual executed components under the declared cash method. An abstract direction label cannot supply those quantities. No count reconciliation alone establishes net profit, a complete risk path or causal model evaluation.

A bounded result for a buyer

Report eligible count, observed outcome count, unresolved states, metric convention and evidence level together. A useful conclusion can reproduce the stated closed-only arithmetic while explaining why a larger advertised claim remains unverified. The source may later supply missing records; retain that as a dated update rather than replacing the original sample. The purpose is an inspectable population, not an invented rating that fills every gap with a preferred provider's marketing.

Completed fictional evidence case

An invented set contains ten issued predictions: four documented positives, two documented negatives and four unresolved. The observed closed-only positive fraction is 4/6, about 66.7%. Four unresolved predictions are 40% of the issued population. Counting those unresolved as wins would invent outcomes; deleting them from a claim about all ten would hide coverage. The completed review retains both counts and marks any all-issued success fraction unestablished. No account return or live provider record is represented, and the small invented sample supports no forecast of future reliability.

Evidence objectRecorded valueClaim boundary
Issued population10Original IDs retained
Resolved population64 positive / 2 negative
Unresolved population4No guessed outcomes
Closed-only fraction4/6Not a universal hit rate
An original claim, its available references and the bounded review
Illustrative record flow; no vendor experiment or customer account is pictured.

Direct sources and reviewed scope

Public documents, not private vendor experiments. The worked cases are invented; source dates do not establish a model-validation engagement.

Related evidence questions