Skip to the evidence
AI Trading SignalsDefinition / Population / Proof

AI claims / Consumer evidence

Overlapping AI Forecasts and the Sample Unit

Distinguish many messages from many independent observations when forecasts share a horizon, instrument or underlying event.

Start with the unit that generated each row

Several forecasts can share a market event or overlapping outcome window. Their message count is not proof of independent statistical evidence. Preserve instrument, decision ID, horizon, target endpoint and any parent event before calculating a summary. The chronological split reference documents ordered samples and a gap parameter, but a sample-count gap does not automatically resolve every overlapping financial label. The actual target construction must decide which boundaries are relevant.

A buyer should also distinguish repeated delivery from distinct predictions. Five channels reposting one output create five observations of distribution, not five independently trained models. A provider may legitimately produce frequent updates, but the claim needs to state whether each update replaces or supplements the earlier forecast. Neither a high message count nor independent-looking repetition establishes sample independence or model quality.

Preserve shared information and endpoints

Record the time interval during which each outcome is measured. If several rows use the same later closing value, keep that common endpoint visible. An analyst can compare the rows under an explicit convention without treating them as unrelated trials. Uncertainty estimates and validation design require a suitable method; this chapter does not invent an effective sample size or a significance level from the number of overlaps.

A split must follow the target, not only its row date

A training row can have a later outcome endpoint than its displayed date. If that endpoint overlaps the evaluation interval, a calendar boundary alone may not establish separation. Retain any purge or gap rule as part of the experiment identity. The site's calendar checker tests one simple declared horizon and is labelled untested. It does not inspect multiple label windows, establish statistical independence or certify that a vendor implemented an adequate split.

What a comparison should retain

Publish row count, original-idea count, update handling, shared endpoint information and unresolved sample definitions. A stronger claim needs the complete original ledger, not a selected chart. Keep a historical research-fill convention separate from actual subscriber execution even when both use the same idea IDs. The review can explain an overlap without declaring every frequent-prediction model invalid; suitability of the evaluation method remains a specific question to resolve with evidence.

Completed fictional evidence case

A fictional system emits five updates during one ten-minute event, all evaluated against the same later endpoint. The record contains five outputs but one shared event under the stated construction. The reviewer cannot declare five independent strategy-validation samples. It preserves each version and asks how the method handles their relationship. Two delivery retries would enlarge delivery count, not original prediction count. No exact independence adjustment is asserted. The example identifies the unit problem without producing a model return, a confidence interval or a fraud allegation.

Evidence objectRecorded valueClaim boundary
Outputs5Frequent updates
Shared endpoint1 declared eventDependence question
Retries2 extra deliveriesNot new predictions
An original claim, its available references and the bounded review
Illustrative record flow; no vendor experiment or customer account is pictured.

Direct sources and reviewed scope

Public documents, not private vendor experiments. The worked cases are invented; source dates do not establish a model-validation engagement.

Related evidence questions