AI evidence glossary
Definitions used by this research desk, with product and evidence boundaries made explicit.
Algorithmic and machine learning
Algorithmic describes a procedure. ML describes a trained component. Automated routing and a language-model explanation need not be the decision model.
Feature availability
A feature's observation date, release time, receipt time and revision history can differ. Causal evaluation asks which value was actually available before a decision.
Holdout and development
An untouched holdout has not influenced experimental choices. Once its results shape a rule, its evidential role changes; renaming it does not restore independence.
Version and retraining
A version identifies a model artefact and its applicable pipeline. Retraining needs a recorded boundary and deployment time rather than retroactively assigning earlier predictions to an improved model.
Backtest and forward record
A backtest is historical simulation under assumptions. A prospective record retains outputs as they occur. Actual account results need executions and costs. These layers cannot substitute for one another.
Receipt and confidence label
A receipt concerns covered bytes and verified attestation, not trained-model quality. A confidence label is not automatically a calibrated probability, winning count or allocation rule.
Sources
- QuantConnect optimization parameters
- ALFRED real-time and vintage data
- OpenTimestamps protocol documentation
Research guides
AI data leakage and holdout audit
Audit feature availability, outcome overlap and test exposure with a frozen experiment record and a worked calendar-boundary example.
Backtest, forward and live evidence
Reconcile simulated decisions, prospective instructions and account fills without promoting one evidence layer into another.
Algorithmic versus AI signal claims
Map fixed rules, trained predictions, AI explanations and order routing to the specific consumer evidence each needs.
Additional model-evidence vocabulary
Calibration: comparison of declared event probabilities with observed frequencies under a stated method. Ranking: ordering outputs against defined classes, not necessarily probability accuracy. Threshold: the selected boundary converting an output into a decision. Vintage: the data version available at a particular historical time. Effective version: the artefact actually bound to an output, not merely the newest file.
Evaluation population: the frozen set eligible for a claimed measure. Unresolved outcome: an eligible observation without its defined endpoint. Shared horizon: overlapping information or outcome windows that require dependence-aware interpretation. Transform contract: the fitted preprocessing and units expected by the model. Explanation provenance: the original sources, versions and clocks behind a narrative. Each appears in a distinct completed consumer case.