Skip to the evidence
AI Trading SignalsDefinition / Population / Proof

Backtest, forward and live evidence

Reconcile simulated decisions, prospective instructions and account fills without promoting one evidence layer into another.

Guide revision and linked reference check: 10 October 2026. Fictional examples; no private model test.

Identify the claim before collecting files

Write the advertised claim in a single sentence with its period, account or model identity and result convention. Historical model performance, prospective signal publication and subscriber profit require different records. If a seller says live, ask whether that means a running software process, an observation stream or actual funded execution. Do not choose the most favourable interpretation on behalf of the seller. Retain the original wording beside the review question.

Start separate folders or ledger fields for each evidence layer. A backtest report belongs with its dataset and execution convention. A prospective record belongs with its publication history. An account statement belongs with deposits, withdrawals, orders and fills. Related records can be reconciled through stable identifiers without being merged into one ambiguous percentage. This separation makes a missing fill or amendment visible. It also prevents a genuine historical simulation from being dismissed simply because it is not something it never claimed to be.

Reproduce the simulation convention, not the sales chart

Ask which instrument histories, universe membership, timezone and corporate-action conventions were used. Retain the model version, entry objective, chosen exit and treatment of absent observations. Costs need units and timing, not just a word such as realistic. A favourable research fill under a declared observation window remains a hypothetical reference; it cannot be described as the price every subscriber obtained. Preserve the frozen convention when comparing releases.

An invented example contains twenty closed hypothetical decisions, fourteen labelled positive and six negative, plus five still open. The closed-only positive fraction is fourteen divided by twenty, or seventy percent. Twenty-five total decisions answer a different coverage question. The example supplies no actual trading result and says nothing about net return, position weights or the final outcomes of the open items. Keep the unresolved rows instead of forcing them into wins or dropping them from the instruction count. A chart alone cannot reveal these denominators.

Count prospective instructions before counting profits

A forward observation should retain decisions when they appear, including those later cancelled, expired, rejected or left unresolved. Record a model deployment change at the time it happens rather than assigning an earlier prediction to a better later artefact. Distinguish source publication from subscriber receipt: a timestamp attached to one does not prove the other. Corrections should link to originals rather than overwrite them invisibly.

Suppose a fictional stream contains twelve messages: four commentaries, six actionable originals and two amendments to those originals. There are six original instructions, not twelve independent trades. If three of those six reached an account and three did not, execution coverage in this toy set is one half. That is not a profitability estimate. The amendment relationship and each unexecuted reason matter. A lost permission, stale instruction or missing acknowledgement belongs to an operating record, even if no profit can yet be calculated for it.

Reconcile execution states without assuming a fill

For every actionable instruction, map the requested order to acknowledgements, partial fills, cancellations and remaining quantity. A timeout is an unknown state until reconciled, not proof that an order never existed. An accepted order is not necessarily an executed order. Keep event ordering and venue identifiers where available, with private account information redacted in a public review. Do not place real orders to demonstrate this guide.

A hypothetical request for one unit can fill 0.4 units and cancel the remaining 0.6. The account still owns 0.4 units; cancellation is not an exit. A later confirmed sale of 0.25 leaves 0.15. These arithmetic states illustrate why a source signal and an account trade can diverge. Fees, financing, currency conversion and mark-to-market conventions must then be applied to the actual quantities and clocks. No generic platform description proves that every provider or subscriber records those states correctly.

Keep cash flow and model identity in the result

An account value can rise after a deposit even when the trading result is flat or negative. Retain externally contributed and withdrawn cash with its timing. Name the return convention used and avoid presenting a raw balance ratio as a cash-flow-adjusted result. An account-level measure and an unweighted sum of model percentages are not interchangeable. If the aggregation method is unavailable, report the published number as an attributed claim rather than independently reproduced performance.

Similarly, a family label can cover several model versions. Preserve the sequence of versions, retirement dates and evaluation rules. A later corrected dataset should not silently alter the old release receipt. If a source changes its record, keep both the old assessment and the dated correction. Do not infer a complete population from one selected successful account. The purpose of reconciliation is to expose what the records support and where they stop, not to manufacture a universally comparable league table.

Write an evidence-layer decision for the reader

End the assessment with the reviewed period, supplied artefacts, missing states and the strongest supported statement. Documentation-only research remains documentation-only even if the interface looks convincing. A preserved prospective stream can support publication chronology without proving funded execution. A reconciled account can establish a specific account experience without forecasting the next subscriber outcome. Say which boundary applies instead of using an unqualified verified label.

Use the checkpoint table to prioritise the next request. A consumer may need an actual cost ledger more urgently than another screenshot of gross profit. Someone comparing research platforms may instead need stable experiment identities and export formats. Preserve private information locally and share only redacted references. These pages have not run a paid model, subscribed a funded account or verified a live return. Their value is a reproducible way to distinguish evidence objects and prevent one valid object from being used to imply several unsupported conclusions.

Review checkpoints

Evidence requests, not observed strategy results
ObjectPreserveDo not infer
SimulationData, fill window, costs and model identityA subscriber received that price
Prospective recordOriginal, amended and unresolved decisionsEvery message became an order
Account outcomeOrders, fills, cash flows and equity basisA screenshot is a complete ledger

Download blank research record (CSV)

Headers only. No trades, account outcomes or operating receipts are supplied.

Sources

Research guides

AI data leakage and holdout audit

Audit feature availability, outcome overlap and test exposure with a frozen experiment record and a worked calendar-boundary example.

Backtest, forward and live evidence

Reconcile simulated decisions, prospective instructions and account fills without promoting one evidence layer into another.

Algorithmic versus AI signal claims

Map fixed rules, trained predictions, AI explanations and order routing to the specific consumer evidence each needs.

Holdout plan date checker | Source-linked assessments

The prospective stream retains unresolved states

A fictional stream has ten original decisions and four unresolved account endpoints. Its hypothetical model outcomes and six documented execution cases require separate populations. Adding the four chart-based results to the account ledger would invent transactions. The complete review can retain the model convention and explain the missing account evidence at the same time.

The original long-form population, timing, cash and quantity examples remain. The new missing-outcomes chapter deepens denominators, while walk-forward identity prevents the newest artefact from silently replacing original output ownership. Research LIVE, deployed code and actually certified subscriber execution are distinct states; this publication does not infer one from another.