Model Selection for Sparse (Robust) Principal Components
These functions compute a suggestion for the sparseness parameter
lambda which is required by function sPCAgrid.
A range of different values for lambda is tested and
according to an objective function, the best solution is selected.
Two different approaches (TPO and BIC) are available, which is further
discussed in the details section.
A graphical summary of the optimization can be obtained by plotting
the function's return value (plot.opt.TPO,
plot.opt.BIC for tradeoff curves or objplot
for an objective function plot).
opt.TPO (x, k.max = ncol (x), n.lambda = 30, lambda.max, ...) opt.BIC (x, k.max = ncol (x), n.lambda = 30, lambda.max, ...)
x |
a numerical matrix or data frame of dimension ( |
k.max |
the maximum number of components which shall be considered for optimizing an objective function (optional). |
n.lambda |
the number of lambdas to be checked for each component (optional). |
lambda.max |
the maximum value of lambda to be checked (optional). If omitted, the lambda which yields "full sparseness" (i.e. loadings of only zeros and ones) is computed and used as default value. |
... |
further arguments passed to |
The choice for a particular lambda is done by optimizing an objective function,
which is calculated for a set of n.lambda models with different
lambdas, ranging from 0 to lambda.max. If lambda.max is not
specified, the minimum lambda yielding "full sparseness" is used.
"Full sparseness" refers to a model with minimum possible absolute sum of
loadings, which in general implies only zeros and ones in the loadings matrix.
The user can choose between two optimization methods:
TPO (Tradeoff Product Optimization; see below), or the
BIC (see Guo et al., 2011; Croux et al., 2011).
The main difference is, that optimization based on the BIC always chooses the
same lambda for all PCs, and refers to a particular choice of k,
the number of considered components.
TPO however is optimized separately for each component, and so delivers
different lambdas within a model and does not depend on a decision on k.
This characteristic can be noticed in the return value of the function:
opt.TPO returns a single model with k.max PCs and
different values of lambda for each PC.
On the contrary opt.BIC returns k.max distinct models
with k.max different lambdas, whereas for each model a different
number of components k has been considered for the optimization.
Applying the latter method, the user finally has to select one of these
k.max models manually,
e.g. by considering the cumulated explained variance,
whereas the TPO method does not require any further decisions.
The tradeoff made in the context of sparse PCA refers to the loss of explained
variance vs. the gain of sparseness. TPO (Tradeoff Product Optimization)
maximizes the area under the tradeoff curve (see plot.opt.TPO),
in particular it maximizes the explained variance multiplied by the number of
zero loadings of a particular component. As in this context the according
criterion is minimized, the negative product is considered.
Note that in the context of sparse PCA, there are two sorting orders of PCs,
which must be considered: Either according to the objective function's value,
(item $pc.noord)or the variance of each PC(item $pc).
As in none-sparse PCA the objective function is identical to the PCs'
variance, this is not an issue there.
The sPCAgrid algorithm delivers the components in decreasing order, according
to the objective function (which apart from the variance also includes sparseness
terms), whereas the method sPCAgrid subsequently re-orders the
components according to their explained variance.
The functions return an S3 object of type "opt.TPO" or "opt.BIC" respectively, containing the following items:
pc |
An S3 object of type |
pc.noord |
An S3 object of type |
x |
The input data matrix as provided by the user. |
k.ini, opt |
These items contain optimization information, as used in
functions |
Heinrich Fritz, Peter Filzmoser <P.Filzmoser@tuwien.ac.at>
C. Croux, P. Filzmoser, H. Fritz (2011). Robust Sparse Principal Component Analysis Based on Projection-Pursuit, ?? To appear.
set.seed (0)
## generate test data
x <- data.Zou (n = 250)
k.max <- 3 ## max number of considered sparse PCs
## arguments for the sPCAgrid algorithm
maxiter <- 25 ## the maximum number of iterations
method <- "sd" ## using classical estimations
## Optimizing the TPO criterion
oTPO <- opt.TPO (x, k.max = k.max, method = method, maxiter = maxiter)
oTPO$pc ## the model selected by opt.TPO
oTPO$pc$load ## and the according sparse loadings.
## Optimizing the BIC criterion
oBIC <- opt.BIC (x, k.max = k.max, method = method, maxiter = maxiter)
oBIC$pc[[1]] ## the first model selected by opt.BIC (k = 1)
## Tradeoff Curves: Explained Variance vs. sparseness
par (mfrow = c (2, k.max))
for (i in 1:k.max) plot (oTPO, k = i)
for (i in 1:k.max) plot (oBIC, k = i)
## Tradeoff Curves: Explained Variance vs. lambda
par (mfrow = c (2, k.max))
for (i in 1:k.max) plot (oTPO, k = i, f.x = "lambda")
for (i in 1:k.max) plot (oBIC, k = i, f.x = "lambda")
## Objective function vs. lambda
par (mfrow = c (2, k.max))
for (i in 1:k.max) objplot (oTPO, k = i)
for (i in 1:k.max) objplot (oBIC, k = i)Please choose more modern alternatives, such as Google Chrome or Mozilla Firefox.