Glossary

Overfitting

When a strategy looks profitable because it was fitted to past data rather than capturing a real edge. Test enough variants and one will look excellent by chance; on new data, it fails. Detecting it is the purpose of validation.

Backtest

A simulation of a strategy's performance on historical data. A useful starting point, not a proof of robustness: a good backtest can hide overfitting. The backtest proposes; validation disposes.

Out-of-sample

Data that played no part in the strategy's design or tuning. Measuring out-of-sample performance is the basic way to check whether an edge is real or just memorized history.

Walk-forward

A validation procedure that moves forward through time: train on one segment, test on the next, repeat. It imitates how the strategy would trade in practice and penalizes those that only work looking backward.

Sharpe ratio

The classic measure of risk-adjusted performance. Informative, but easy to inflate: pick the best of many configurations and its Sharpe is biased by the search. That's why it gets "deflated" (see Deflated Sharpe Ratio).

Deflated Sharpe Ratio (DSR)

A corrected version of the Sharpe ratio that discounts the effect of testing multiple variants and the non-normality of returns. It estimates whether the edge is real or an artifact of selection. Published by Marcos López de Prado (2014).

Probability of Backtest Overfitting (PBO)

Estimates how likely it is that the "winning" in-sample configuration turns out mediocre out of sample. A high probability is the signature of overfitting: in-sample performance that doesn't survive once the data changes.

Purged cross-validation (CPCV)

Combinatorial purged cross-validation. It tests the strategy across many combinations of training and testing segments, instead of a single split, for a more stable reading of out-of-sample performance.

Embargo and purging

Two safeguards that keep cross-validation honest with time series. Purging removes training observations that overlap in time with the test segment; the embargo discards a margin around it. Together they prevent nearby information from leaking in and exaggerating the results.

Robustness Score

A 0–100 rating that summarizes how robust a strategy is according to the validation. It comes with a band and a verdict. It measures robustness, not expected profitability, and it does not predict gains. The cutoffs that separate one level from the next are part of the calibration and are not published.

Robustness bands

The categories in which the result is expressed. The band is public; the calibration that determines it is private:

Robust — meets audit-grade standards across the dimensions evaluated.

Conditional Robustness — shows robust potential, with specific weaknesses to refine before deployment.

Limited Robustness — quality below the robustness threshold, with material weaknesses; structural changes are required.

Not Robust — not deployable in its current form; significant changes are recommended before revalidating.

Disclaimer of Opinion

The outcome when the sample is insufficient for an audit-grade evaluation. It is not a «failure» of the strategy: it means there isn't enough evidence to issue a verdict. The path forward is to generate more trade data and resubmit.

Verdict

The certificate's conclusion: the band, the Robustness Score where applicable, and an explanation of what the result means and what the next steps are.

Capped result

When one or more control gates limit the result, the certificate flags it as capped and does not issue a numeric score. It states which control was determinant, without revealing its thresholds.

Certificate ID

The unique identifier of each certificate, in the format TRV-XXXX-XXXX. It lets you refer to a specific validation and, together with the hash, verify its authenticity.

Reproducibility Hash

A cryptographic fingerprint of the verdict and the inputs that produced it. The same inputs produce the same hash, which makes it possible to confirm the certificate wasn't altered after it was issued.

See how these concepts are applied in the methodology →