Getting Started with spfcICOMP
Introduction
The spfcICOMP package implements Shrinkage Principal
Fitted Components (SPFC) methodology for sufficient dimension reduction
under high-dimensional settings.
The package combines
- shrinkage covariance estimation,
- principal fitted components,
- information-complexity model selection,
- adaptive feature screening, and
- downstream predictive modelling
within a unified framework.
Simulate data
Fit SPFC
fit <- spfc_fit(
X = X,
y = y,
d = 1,
ytype = "continuous",
cov_method = "mec",
nslices = 5,
poly_degree = 2
)
fit##
## Shrinkage Principal Fitted Components Fit
## =========================================
##
## Response type: continuous
## Covariance method: mec
## Structural dimension: 1
## Shrinkage rho: 0.334192
## Slices used: 5
##
## Leading eigenvalues:
## [1] 1.832026 0.430107 0.000000 0.000000 0.000000 0.000000
##
## Reduced score matrix dimension:
## [1] 100 1##
## Summary of Shrinkage Principal Fitted Components Fit
## ====================================================
##
## Response type: continuous
## Covariance method: mec
## Structural dimension: 1
## Shrinkage rho: 0.334192
## Slices used: 5
##
## Preprocessing:
## Centred: TRUE
## Scaled: FALSE
##
## Score matrix dimension:
## [1] 100 1
##
## Eigenvalue summary:
## component eigenvalue proportion cumulative_proportion
## 1 1 1.8320256 0.8098666 0.8098666
## 2 2 0.4301068 0.1901334 1.0000000
## 3 3 0.0000000 0.0000000 1.0000000
## 4 4 0.0000000 0.0000000 1.0000000
## 5 5 0.0000000 0.0000000 1.0000000
## 6 6 0.0000000 0.0000000 1.0000000
## 7 7 0.0000000 0.0000000 1.0000000
## 8 8 0.0000000 0.0000000 1.0000000
## 9 9 0.0000000 0.0000000 1.0000000
## 10 10 0.0000000 0.0000000 1.0000000Extract estimated directions
## dir1
## [1,] -0.1615488781
## [2,] 0.0961927812
## [3,] 0.7529084381
## [4,] 0.4137419377
## [5,] 0.1831993803
## [6,] 0.1088300663
## [7,] 0.0166737966
## [8,] -0.0077064971
## [9,] 0.0219785478
## [10,] -0.0535684391
## [11,] 0.0464507770
## [12,] 0.0135626166
## [13,] -0.0398501461
## [14,] -0.1039222163
## [15,] -0.0776381586
## [16,] 0.0361675259
## [17,] 0.0572101996
## [18,] 0.0196295812
## [19,] -0.0209828212
## [20,] -0.0378015513
## [21,] 0.0729443417
## [22,] -0.0418411849
## [23,] -0.0116996981
## [24,] 0.0210767615
## [25,] 0.1252598074
## [26,] -0.0214393605
## [27,] 0.0419656390
## [28,] 0.1308447555
## [29,] -0.0295170665
## [30,] -0.0570300761
## [31,] 0.0877176433
## [32,] 0.0007593785
## [33,] 0.0828312254
## [34,] 0.0327028964
## [35,] -0.0076872575
## [36,] -0.0361780360
## [37,] -0.0213063183
## [38,] 0.0166143165
## [39,] 0.1127124525
## [40,] 0.1340190564
## [41,] 0.1552736802
## [42,] 0.0462433894
## [43,] -0.0665783909
## [44,] -0.0272594779
## [45,] -0.0978535506
## [46,] 0.0209104783
## [47,] 0.0868273682
## [48,] 0.0345079364
## [49,] 0.0594983369
## [50,] 0.0316847716Obtain reduced scores
## SPFC1
## [1,] 2.1140852
## [2,] 2.1653640
## [3,] -2.3247689
## [4,] 0.2069984
## [5,] -0.0555394
## [6,] 2.5517066Predict new observations
## SPFC1
## [1,] 2.1140852
## [2,] 2.1653640
## [3,] -2.3247689
## [4,] 0.2069984
## [5,] -0.0555394Structural dimension selection
dsel <- spfc_select_dimension(
X = X,
y = y,
d_grid = 1:3,
cov_method = "mec",
ytype = "continuous"
)## Scoring d = 1 using covariance method: mec## Scoring d = 2 using covariance method: mec## Scoring d = 3 using covariance method: mec## cov_method ytype d rho nslices_used reduced_model loglik npar
## 1 mec continuous 1 0.334192 5 lm -82.71730 3
## 2 mec continuous 2 0.334192 5 lm -76.04579 4
## 3 mec continuous 3 0.334192 5 lm -75.08006 5
## AIC BIC CAIC ICOMP_IFIM ICOMP_MISSPEC CICOMP
## 1 171.4346 179.2501 182.2501 165.8845 165.7711 182.6124
## 2 160.0916 170.5123 174.5123 152.6739 152.4379 174.8718
## 3 160.1601 173.1860 178.1860 151.4111 151.2615 179.2894## criterion selected_d minimum_value
## 1 AIC 2 160.0916
## 2 BIC 2 170.5123
## 3 CAIC 2 174.5123
## 4 ICOMP_IFIM 3 151.4111
## 5 ICOMP_MISSPEC 3 151.2615
## 6 CICOMP 2 174.8718Feature screening
The default C1F-calibrated feature-screening rule uses the covariance of the fitted downstream reduced model. The reduced-space model is therefore fitted before C1F-based screening is applied.
reduced_model <- fit_reduced_model(
Z = scores,
y = y,
ytype = "continuous"
)
vsel <- spfc_select_variables(
fit = fit,
method = "adaptive_weighted_l1",
selection_rule = "c1f",
reduced_model = reduced_model
)
head(vsel)## variable importance weight penalty shrunk_importance selected
## 1 3 0.7529084 9.710836e-07 0.0000485422 0.7528599 TRUE
## 2 4 0.4137419 3.215747e-06 0.0001607477 0.4135812 TRUE
## 3 5 0.1831994 1.640185e-05 0.0008198903 0.1823795 TRUE
## 4 1 0.1615489 2.109274e-05 0.0010543769 0.1604945 TRUE
## 5 41 0.1552737 2.283207e-05 0.0011413217 0.1541324 TRUE
## 6 40 0.1340191 3.064840e-05 0.0015320420 0.1324870 TRUEBenchmark covariance estimators
bench <- benchmark_spfc(
X = X,
y = y,
d = 1,
methods = c(
"mec",
"oas",
"sre",
"sde",
"cse"
),
verbose = FALSE
)
bench##
## SPFC Benchmark
## ==============
##
## Response type: continuous
## Structural dimension: 1
## Variable method: adaptive_weighted_l1
## Validation: resubstitution
## Methods compared: mec, oas, sre, sde, cse
##
## Performance summary:
## cov_method d reduced_model runtime_sec rmse mae n_selected
## 1 mec 1 lm 0.008384466 0.5533495 0.4454105 32
## 2 oas 1 lm 0.004213572 0.5636975 0.4563440 32
## 3 sre 1 lm 0.003581047 0.3675212 0.2958654 33
## 4 sde 1 lm 0.003173590 0.6169820 0.5004842 34
## 5 cse 1 lm 0.003554106 0.4649883 0.3779666 34
##
## Applicable covariance diagnostics:
## cov_method diagnostic value
## 1 mec rho 0.3341920
## 2 oas rho 0.1940417
## 3 sre rho 0.1000000
## 4 cse rho 0.5000000
## 5 mec nslices_used 5.0000000##
## Summary of SPFC Benchmark
## =========================
##
## Response type: continuous
## Structural dimension: 1
## Variable method: adaptive_weighted_l1
## Validation: resubstitution
## Selection metric: rmse
## Smaller is better: TRUE
## Best method: sre
##
## Benchmark table:
## cov_method d reduced_model runtime_sec rmse mae n_selected
## 1 mec 1 lm 0.008384466 0.5533495 0.4454105 32
## 2 oas 1 lm 0.004213572 0.5636975 0.4563440 32
## 3 sre 1 lm 0.003581047 0.3675212 0.2958654 33
## 4 sde 1 lm 0.003173590 0.6169820 0.5004842 34
## 5 cse 1 lm 0.003554106 0.4649883 0.3779666 34
##
## Applicable covariance diagnostics:
## cov_method diagnostic value
## 1 mec rho 0.3341920
## 2 oas rho 0.1940417
## 3 sre rho 0.1000000
## 4 cse rho 0.5000000
## 5 mec nslices_used 5.0000000Simulation study
results <- run_spfc_simulation(
response_type = "continuous",
nrep = 5,
n = 100,
p = 50,
d = 1,
s = 5,
rho_x = 0.5,
snr = 2,
cov_methods = c(
"mec",
"oas"
)
)
summary_results <-
summarise_spfc_simulation(
results
)
summary_results## ytype cov_method criterion selection_rule variable_method n p
## 1 continuous mec KNOWN_D c1f adaptive_weighted_l1 100 50
## 2 continuous mec KNOWN_D c1f c1f_extension 100 50
## 3 continuous oas KNOWN_D c1f adaptive_weighted_l1 100 50
## 4 continuous oas KNOWN_D c1f c1f_extension 100 50
## true_d s rho_x snr nrep dimension_recovery_rate mean_selected_d sd_selected_d
## 1 1 5 0.5 2 5 1 1 0
## 2 1 5 0.5 2 5 1 1 0
## 3 1 5 0.5 2 5 1 1 0
## 4 1 5 0.5 2 5 1 1 0
## mean_runtime_sec sd_runtime_sec mean_subspace_distance sd_subspace_distance
## 1 0.006051683 0.0002952715 0.6206472 0.1182854
## 2 0.006636333 0.0010250420 0.6206472 0.1182854
## 3 0.002852201 0.0001224630 0.6252777 0.1161637
## 4 0.002832556 0.0001020159 0.6252777 0.1161637
## mean_precision sd_precision mean_recall sd_recall mean_f1_variable
## 1 0.1524746 0.013626567 1 0 0.2644103
## 2 0.1524746 0.013626567 1 0 0.2644103
## 3 0.1537493 0.008417082 1 0 0.2664472
## 4 0.1537493 0.008417082 1 0 0.2664472
## sd_f1_variable mean_n_selected sd_n_selected mean_c1f
## 1 0.02046288 33.0 2.915476 0.1978165
## 2 0.02046288 33.0 2.915476 0.1978165
## 3 0.01267508 32.6 1.816590 0.2039322
## 4 0.01267508 32.6 1.816590 0.2039322
## mean_c1f_complexity_fraction mean_c1f_loading_scale mean_c1f_hd_factor
## 1 0.1623566 0.04774634 0.279715
## 2 0.1623566 0.04774634 0.279715
## 3 0.1666243 0.04797230 0.279715
## 4 0.1666243 0.04797230 0.279715
## mean_c1f_complexity_multiplier mean_c1f_global_penalty mean_rmse sd_rmse
## 1 1.162357 0.01538218 0.6409285 0.1216673
## 2 1.162357 0.01538218 0.6409285 0.1216673
## 3 1.166624 0.01549119 0.6437596 0.1228059
## 4 1.166624 0.01549119 0.6437596 0.1228059
## mean_mae sd_mae mean_accuracy sd_accuracy mean_sensitivity
## 1 0.5046014 0.09045208 NA NA NA
## 2 0.5046014 0.09045208 NA NA NA
## 3 0.5060441 0.08858848 NA NA NA
## 4 0.5060441 0.08858848 NA NA NA
## sd_sensitivity mean_specificity sd_specificity mean_f1_classification
## 1 NA NA NA NA
## 2 NA NA NA NA
## 3 NA NA NA NA
## 4 NA NA NA NA
## sd_f1_classification
## 1 NA
## 2 NA
## 3 NA
## 4 NAPlotting
References
Cook, R. D. and Forzani, L. (2008). Principal Fitted Components for dimension reduction in regression. Statistical Science, 23(4), 485–501. doi:10.1214/08-STS275.
Chen, Y., Wiesel, A., Eldar, Y. C. and Hero, A. O. (2010). Shrinkage algorithms for MMSE covariance estimation. IEEE Transactions on Signal Processing, 58(10), 5016–5029. doi:10.1109/TSP.2010.2053029.
Bozdogan, H. (2000). Akaike’s Information Criterion and recent developments in information complexity. Journal of Mathematical Psychology, 44(1), 62–91. doi:10.1006/jmps.1999.1277.
Olorede, K. O. and Yahya, W. B. (2019). A new covariance estimator for sufficient dimension reduction in high-dimensional and undersized sample problems. arXiv. doi:10.48550/arXiv.1909.13017.