AI claims / Consumer evidence
AI Feature Importance Is Not a Causal Explanation
Keep a fitted model’s sensitivity measure, its actual decision inputs and a later narrative separate.
What the importance measure actually asked
An importance measure depends on a fitted model, dataset and metric. It does not automatically identify a causal market mechanism or the reason for a particular live trade. scikit-learn's permutation-importance reference discusses model- and dataset-dependent interpretation, including correlated features. Retain the definition and evaluation context before converting a plotted bar into a universal explanatory claim. A seller's named reasons may be a different object from the features used by the decision engine.
A weak or unvalidated model can still produce an importance graphic. The graphic does not establish its predictive skill. Ask first what evaluated model and observation population underlie it, then which question the chosen importance method answers. A readable feature explanation can be useful without demonstrating that changing the real-world variable would cause the forecasted outcome.
Correlated inputs complicate simple narratives
Several features can contain related information. A measure of one input can change when another correlated input is present, absent or transformed. Preserve feature set and preprocessing version alongside the report. Do not interpret a small bar as proof the corresponding phenomenon never matters. Conversely, a large bar is not proof of an independent causal driver. This publication does not infer hidden vendor architecture from a public screenshot of selected feature names.
A language explanation needs its own provenance
If a narrative is generated after a numeric decision, preserve its timestamp and relationship to the original inputs. Its persuasive reasons are not automatically a faithful description of the decision engine. Request a documented mapping if the product claims that fidelity. An explanation corrected after an outcome should remain a new version, not become the original contemporaneous rationale. This chapter reviews observable records rather than claiming access to hidden model reasoning.
How to write a bounded interpretation
Name the method, model, data period, metric and inspected output. State whether the report concerns global model behaviour or one supplied instance. Leave causal or runtime claims unsupported when the records do not establish them. A buyer can compare explanation functions and auditability without an invented profitability score. The original algorithmic-versus-AI protocol preserves component roles; this chapter deepens the narrower attribution question with a completed fictional mismatch.
Completed fictional evidence case
A fictional model uses features A and B, while a later generated paragraph attributes its decision to C. No record connects C to the fitted input contract. The review can preserve the paragraph as an explanation product, but cannot label C an independently verified causal driver or model input. A permutation report on A and B would answer a different documented question. The case contains no actual model test, vendor accusation or economic causal conclusion; the missing mapping is the precise evidence request.
| Evidence object | Recorded value | Claim boundary |
|---|---|---|
| Fitted feature contract | A and B | Declared synthetic inputs |
| Later narrative | C | Separate explanation |
| Claimed causal reason | Unestablished | No mapping supplied |

Direct sources and reviewed scope
Public documents, not private vendor experiments. The worked cases are invented; source dates do not establish a model-validation engagement.