Skip to content

Outputs & Diagnostics

Every execution of Risk Bridge writes a self-contained, timestamped run directory under the specified --output-root:

<output_root>/<timestamp>_<run_label>/
├── intermediate/
└── final/

The output contract follows schema version 1.1.0 (defined in risk_bridge.output_schema).

Final Artifacts (final/)

1. final/run_metadata.csv

Captures the complete configuration and environment of the run.

  • run_id: Unique timestamped identifier.
  • scenario: Scenario index (\(1, 2, 3\)) or user-data.
  • nsim, samplesize, Ntarget, Nsource, Nreference: Sample sizes.
  • fpr_target: Configured target FPR for threshold selection.
  • calibration_tolerance: Constraint tolerance \(\epsilon\) used during optimization.
  • schema_version: Version string of the output contract (1.1.0).

2. final/fit_diagnostics.csv

Contains numerical solver status and feasibility diagnostics for every iteration and sampling path.

Column Description
iter Iteration index (\(0, \dots, \text{nsim}-1\)).
path Analysis path: PSM (Propensity Score Matched) or RS (Random Sample).
mle_success Boolean: Whether unconstrained BFGS optimization converged.
cmle_success Boolean: Whether constrained cMLE optimization converged and satisfied constraints.
mle_status Status message returned by SciPy for unconstrained MLE.
cmle_status Status message returned by SciPy for constrained cMLE.
mle_objective Final negative log-likelihood value for unconstrained MLE.
cmle_objective Final negative log-likelihood value for constrained cMLE.
cmle_max_violation Maximum absolute calibration constraint violation: \(\max_k \lvert g_k(\hat{\theta}) \rvert\).

[!NOTE] A cMLE fit is declared successful (cmle_success == True) if and only if cmle_max_violation <= calibration_tolerance.

3. Parameter Estimates: final/est_cml_psm.csv, final/est_ml_psm.csv

Contains fitted parameter values for \(\theta = (\alpha, \beta_X, \beta_Z, \gamma_0, \gamma_X, \sigma)\):

  • alpha: Intercept of the outcome logistic model.
  • beta_X1, beta_X2, ...: Coefficients for baseline covariates \(X\) in outcome model.
  • beta_Zcat: Coefficient for categorical risk stratum \(Z_{\text{cat}}\) in outcome model.
  • gamma_0: Intercept of the truncated-lognormal mean parameter \(\tau(X)\).
  • gamma_X1, ...: Coefficients for \(X\) in the truncated-lognormal mean parameter \(\tau(X)\).
  • gamma_sigma: Scale parameter \(\sigma > 0\) of the truncated-lognormal distribution.

4. Calibration Metrics: final/calibration_metrics.csv

Reports primary statistical calibration performance evaluated on the target cohort:

  • calibration_in_the_large (CITL): Intercept from logistic recalibration (\(\text{logit}(Y) = a + \text{logit}(\hat{p})\)). Ideal value is \(0\).
  • calibration_slope: Slope from logistic recalibration (\(\text{logit}(Y) = a + b \cdot \text{logit}(\hat{p})\)). Ideal value is \(1.0\).
  • observed_expected_ratio (O/E): Ratio of total observed events to total predicted events \(\sum Y_i / \sum \hat{p}_i\). Ideal value is \(1.0\).
  • brier_score: Mean squared error of probability predictions \(\frac{1}{N} \sum (Y_i - \hat{p}_i)^2\). Lower is better.

5. final/calibration_residuals.csv

Provides granular post-fit moment residuals for each risk interval \(k=1,\dots,K\):

  • risk_interval: The stratum index.
  • expected_risk: Model-implied risk in interval \(k\) under fitted parameters \(\hat{\theta}\).
  • p_external: Reference population external benchmark risk in interval \(k\).
  • residual: Difference \(\text{expected\_risk} - p_{\text{external}}\).

6. Discrimination: final/roc_metrics.csv

Reports Area Under the ROC Curve (AUC) for: - roc_CML_PSM: cMLE on propensity-score matched sample. - roc_ML_PSM: Unconstrained MLE on propensity-score matched sample. - roc_CML_RS: cMLE on random source sample. - roc_ML_RS: Unconstrained MLE on random source sample. - roc_base: Base risk model \(\phi(X)\) fitted on reference data only.

7. Classification: final/accuracy_metrics.csv

Evaluates binary classification performance at the decision threshold corresponding to the target FPR: - TPR: True Positive Rate (Sensitivity). - PPV: Positive Predictive Value (Precision). - TNR: True Negative Rate (Specificity).

8. Reproducibility Sidecar: final/environment.json

Records: - Package version (1.0.5), Git commit SHA, Python version, platform architecture. - Thread configuration (OMP_NUM_THREADS, MKL_NUM_THREADS). - Formal reproducibility contract with numerical tolerances (rtol=1e-6, atol=1e-8).