Give a result something difficult to survive.
A useful framework makes the assumptions visible. It separates software correctness, statistical evidence and executable trading performance instead of treating them as the same achievement.
One statistic
is not a decision.
The order matters: establish whether the information was available and the experiment was valid before interpreting its score.
Inspect the public code ↗The Monolith research workflow brings together probabilistic inference, search adjustment, overfitting analysis and implementation checks. These are complementary tools—not interchangeable stamps of approval.
The public code is a technical work sample. This website does not run the full validation pipeline in your browser, certify a strategy or produce a deployment decision.
The useful output is not a reassuring label. It is a decision another researcher can inspect.
- Point-in-time integrity
Could the decision have been made then?
Inspect availability timestamps, revisions, universe membership and train/test separation. No statistical correction can rescue data that leaked the answer.
- Probabilistic Sharpe
How uncertain is the estimate?
Consider sample size, skew and kurtosis when assessing a Sharpe estimate against a specified benchmark. The observation frequency and assumptions matter.
- Deflated Sharpe
What search produced the winner?
Account for the selection process and the number of effective trials. A standalone headline ratio cannot tell you how many alternatives were tried.
- Backtest overfitting
Does selection generalise?
Use combinatorial cross-validation where the experiment structure permits it. PBO asks a different question from a conventional confidence interval.
- Costs & capacity
Does the effect survive implementation?
Stress spread, slippage, financing, borrow, turnover and feasible size. Measure costs in the same market conditions in which the proposed effect appears.
- Exposure & benchmarks
What actually explains the return?
Compare against suitable passive and simple active alternatives. Separate a market exposure, a compensated premium and a claim of incremental alpha.
- Untouched evaluation
Was the final test really final?
Preserve the evaluation window and record the search history. Reopening the holdout to change the model converts it into training information.
Review the rejection.
Not just the winner.
A pessimistic implementation defect can make a real effect look worse. Rejections deserve the same challenge as positive findings: inspect the statistic, test known cases and consider the alternative explanation.
Likewise, a correct accounting reconciliation establishes that saved returns and equity agree. It does not prove the data are point-in-time, the execution model is faithful or the return is repeatable.
See the archived-system case study ↗