Which optimizer target should we trust out of sample?
We tested whether the score that looks best during optimization actually points to portfolios with better future paths.
Educational Risk Disclosure
This presentation is for education and research process review only. It is not personalized investment, trading, legal, tax, or financial advice.
Trading Risk
- Trading and investing involve risk.
- You can lose principal.
- No strategy or model guarantees profits.
Evidence Limits
- Backtests and holdouts depend on assumptions and historical data.
- Validation reduces false confidence; it does not remove uncertainty.
- Future market behavior can differ materially from tested periods.
Use Scope
- The examples are research artifacts, not trade instructions.
- Objective rankings are evidence for review, not a recommendation to buy or sell.
- Position sizing, taxes, suitability, and execution constraints are out of scope.
A good final return can hide a bad ride.
The highlighted windows finish with similar median returns, but very different path pain.
We rewarded smooth compounding, not just bigger returns.
Plain English
- Grow capital out of sample.
- Avoid long, painful drawdowns.
- Keep max drawdown as a veto/diagnostic, not the whole score.
How well the portfolio actually grows over the future path.
How deep and persistent the drawdowns are along the way.
The main score is geometric Martin: future compounding divided by drawdown pain.
Scoring ruleWhich targets produced the best future paths?
This now includes geometric Martin as an optimizer target, not just as the OOS scoring rule.
Did the ranking hold up when the future test got longer?
Top is best. The direct geometric-Martin objective did not stay ahead once the test period got longer.
Did the optimizer score predict better future paths?
This asks whether higher in-sample optimizer scores actually pointed to better future path quality.
The longer future test made the signal clearer.
A target can look okay recently and much more decisive over a longer future period.
Did the result still make sense on pure growth?
This checks whether the same optimizer scores also line up with future growth.
Did the result still make sense after looking at bad tails?
This checks whether the optimizer score also lines up with cleaner upside/downside behavior.
Put the evidence into one stability ranking.
The final ranking combines future path quality, predictive power, and consistency. Lower is better.
Why does the longer holdout rank them this way?
The direct geometric-Martin objective had a decent IC, but weaker longer-period path rank and consistency.
What should we take away?
Bottom line: use geometric Martin as the out-of-sample judging rule, but do not assume the geometric-Martin optimizer is the best target. Sharpe and VI remain the current finalists.
Conclusion