Risk model construction: the Σ matrix, the factor loadings, and why both matter

Σ — the covariance matrix the optimizer reads — is the most consequential single object on the platform. It defines the tracking-error metric, the risk-budget interpretation of the portfolio, and the shape of the dispersion the optimizer can spend. Build Σ wrong and every downstream number is wrong. This post is about how it's built, why we use Ledoit–Wolf shrinkage instead of a rolling sample covariance, and how the factor loadings B are derived alongside it.

For an N-name universe, the sample covariance matrix has N(N+1)/2 unique entries. The information you have to fit it is N × T observations of returns. With N = 500 and T = 252 (one year of daily returns), you're estimating 125,250 parameters from 126,000 data points. The result is statistically degenerate — Σ̂ is nearly singular, eigenvalues are dispersed across many orders of magnitude, and small inversion errors blow up downstream.

Three symptoms when the optimizer reads a near-singular Σ:

• Realised TE drifts wildly off the constraint. The TE budget becomes meaningless because the metric it's measured in has collapsed in some directions.
• The optimizer concentrates risk in a small number of names whose estimated covariances happen to be near-zero. These are usually the names with thin price history, not the names with low real risk.
• Day-over-day weight churn rises. A small change in the observed return panel produces a large change in Σ̂⁻¹, which the optimizer faithfully transmits to weight changes.

Σ̂_LW  =  (1 − α) · Σ̂_sample   +   α · F

where  Σ̂_sample = sample covariance from the trailing T-day return panel
       F        = shrinkage target (usually a scaled identity matrix)
       α        ∈ [0, 1], chosen by Ledoit–Wolf optimal-shrinkage formula

       F = (1/N) · trace(Σ̂_sample) · I

The intuition: pull every eigenvalue of Σ̂_sample toward the mean eigenvalue. The very small ones (the spurious near-zero directions) get pulled up; the very large ones (overfit to a recent regime) get pulled down. The result is well-conditioned and inverts cleanly. The shrinkage intensity α is computed analytically — no hyperparameter search.

The trade-off: shorter windows track regime changes faster but produce noisier estimates; longer windows are more stable but lag turning points. We benchmarked four window sizes against held-out data:

Alongside Σ, the platform keeps a factor-loadings matrix B derived from a linear regression of each name's returns on six factor proxies:

The risk model is rebuilt nightly by a separate process from the DailyRunner. Each rebuild produces a versioned snapshot (today's Σ + B + window endpoints) that the runner reads at the start of each account solve. The snapshot is persisted with the Run row as sigma_blob and loadings_blob, so a replay a year later uses the exact matrices the original solve saw — not whatever today's risk model would produce.

• Asymmetric tails. Σ assumes symmetric Gaussian-like dispersion. Real returns have fat left tails and skew. For tail-risk-relevant decisions (concentrated positions, options overlay) the optimizer's TE estimate underprices crash risk.
• Time-varying correlations. The 504-day window blends correlations across regimes; a "correlation goes to one in a crisis" event isn't reflected until it's already happened. This is a classical unconditional-Σ limitation, not specific to the platform.
• Non-equity factors. The factor list above is equity-only. Strategies that interact with bonds, commodities, or FX would need additional factors and (likely) a different shrinkage target.

For how Σ is consumed downstream during replacement-security selection, see replacement-security selection. For how the snapshot mechanism preserves the risk model on each Run, see snapshot reproducibility.

Methodology note · risk-model choices propagate to every downstream number. Worth getting right.