The elastic net adds the ridge penalty \(\frac{\lambda_2}{2} \|\boldsymbol\beta\|_2^2\) to the lasso. Tikhonov regularization in general form [1], [2] replaces it by \(\frac{\lambda_2}{2} \|\boldsymbol\Gamma \boldsymbol\beta\|_2^2 = \frac{\lambda_2}{2} \boldsymbol\beta^\top \boldsymbol K \boldsymbol\beta\) with \(\boldsymbol K = \boldsymbol\Gamma^\top \boldsymbol\Gamma\). The matrix encodes what the analyst knows about the coefficients. \(\boldsymbol K = \boldsymbol I\) gives the ridge penalty of the elastic net, a first-difference operator \(\boldsymbol\Gamma\) gives the graph Laplacian of a chain and asks neighbouring coefficients to be equal (smoothness), a graph Laplacian in general [3], [4] couples the variables along the edges of a graph, a group-mean matrix reproduces the penalty of the informed elastic net, and a block-diagonal centering matrix asks the members of a group to share a coefficient. The Tikhonov T-Rex selector runs the terminating Tikhonov-net solver TIKNET, a pathwise homotopy algorithm, with such a matrix inside the random experiments.
The active variables are a contiguous block with equal coefficients, the case a chain Laplacian is made for.
set.seed(2718)
n <- 120
p <- 60
true_support <- 21:30
beta <- numeric(p)
beta[true_support] <- 1
X <- matrix(rnorm(n * p), n, p)
y <- as.numeric(X %*% beta + rnorm(n))trex_gamma_to_k() builds \(\boldsymbol K\) from an operator \(\boldsymbol\Gamma\) and returns a sparse
matrix of the Matrix package. A dense base matrix is accepted everywhere
a penalty matrix is.
gamma <- diff(diag(p)) # first differences, (p - 1) x p
if (requireNamespace("Matrix", quietly = TRUE)) {
K <- trex_gamma_to_k(gamma)
class(K)
} else {
K <- crossprod(gamma)
}
#> [1] "dgCMatrix"
#> attr(,"package")
#> [1] "Matrix"
K[1:4, 1:4]
#> 4 x 4 sparse Matrix of class "dgCMatrix"
#>
#> [1,] 1 -1 . .
#> [2,] -1 2 -1 .
#> [3,] . -1 2 -1
#> [4,] . . -1 2The Tikhonov control is required, because it carries \(\boldsymbol K\). The penalty weight is
fixed here. A base control that is passed has to name the solver of the
selector, "TIKNET". Without a base control the selector
runs the defaults of trex_control() with that solver.
ctrl <- trex_control(solver = "TIKNET", K = 10)
tik <- TRexTikhonovSelector$new(X, y, tFDR = 0.1, seed = 4, verbose = FALSE,
tik_control = trex_tikhonov_control(tikhonov_K = K, lambda_2 = 1),
control = ctrl)
tik$select()
tik$selected_indices
#> [1] 21 22 23 24 25 26 27 28 29 30
trex_fdp(tik$selected_indices, true_support)
#> [1] 0
trex_tpp(tik$selected_indices, true_support)
#> [1] 1
tik$lambda2_used
#> [1] 1TRexTikhonovSelector inherits the fields of
TRexSelector and adds lambda2_used,
lambda2_cv_diagnostics, lambda2_pilot and
dummy_clusters. A selector runs once, as every selector
does.
lambda_2 < 0 (the default) hands the choice to
lambda2_method. Three rules exist.
"PILOT_KNEE" (the default) reads \(\lambda_2\) off a few random experiments
per candidate of pilot_grid, by the selector’s own
principle: the smallest \(\lambda_2\)
whose pilot selection count is within pilot_knee_tolerance
of the largest, the knee of the profile. pilot_K,
pilot_T and pilot_L_multiplier size the pilot,
pilot_statistic names the count ("MAJORITY",
"CALIBRATED", "GROUP_MAJORITY"). The method
run_pilot_only() runs the pilot without the selection and
returns its record."CV_1SE_IEN_CCD", "CV_MIN_IEN_CCD",
"CV_1SE_TIK_SVD" and "CV_MIN_TIK_SVD".
cv_n_folds, cv_n_lambda1,
cv_n_lambda2, cv_dfmax_ratio and
cv_seed control them. They judge a fitted estimate, so on
structured designs they tend to pick the edges of their grid.trex_tikhonov_lambda2_power_rule() runs one full
selection per grid value with common dummies and takes the knee of that
profile. It is the reference rule behind the pilot, at the cost of one
selection per grid value, and it returns every probe’s selection.
trex_gvs_lambda2_power_rule() is the same rule on the
T-Rex+GVS selector.Without a grid the rules run on
trex_default_lambda2_grid(), the grid the C++ core holds
for them.
A supplied lambda_2 is used as it is, on the scale of
the standardized columns. A grid, pilot_grid or the grid of
a power rule, is stated for columns of unit norm. Under
scaling_mode = "ZSCORE" the columns have the squared norm
\(n - 1\), so the rules run at the grid
times \(n - 1\), and one grid means the
same shrinkage under every scaling mode. Every rule reports its pick on
the scale of a supplied value, so lambda2_used can be
passed as lambda_2.
pilot <- TRexTikhonovSelector$new(X, y, tFDR = 0.1, seed = 4, verbose = FALSE,
tik_control = trex_tikhonov_control(
tikhonov_K = K,
pilot_grid = c(0, 1, 10), pilot_K = 4, pilot_T = 2),
control = ctrl)
pilot$select()
pilot$lambda2_used
#> [1] 0
pilot$lambda2_pilot$grid
#> [1] 0 1 10
pilot$lambda2_pilot$counts
#> [1] 13 11 10The false discovery rate (FDR) guarantee needs dummies that are
exchangeable with the null variables under the penalty. Two mechanisms
take care of that. cluster_dummies = TRUE (default) draws
the dummies as per-cluster coloured layers on a correlation clustering
of the variables (prior_groups, or hierarchical clustering
with corr_max and hc_linkage), and every dummy
joins the \(\boldsymbol K\)-coupling of
the variable it mirrors (the folded coupling).
cluster_dummies = FALSE exists for validation runs. The
field dummy_clusters reports the clusters of a run.
penalty_structure = "GROUP_CENTERING" lets the selector
build \(\boldsymbol K\) itself as the
block-diagonal centering penalty \(\boldsymbol
I - \boldsymbol J / p_m\) over the groups of
prior_groups, so the members of a group share a
coefficient. The structure needs prior_groups (group
numbers from 1, one per variable), and the selector refuses it without
them. An all-singleton partition is legitimate, the penalty vanishes and
the path is the lasso on per-variable cluster dummies.
trex_build_group_centering_k() returns this matrix for a
group assignment.
groups <- rep(1:6, each = 10)
gc <- TRexTikhonovSelector$new(X, y, tFDR = 0.1, seed = 4, verbose = FALSE,
tik_control = trex_tikhonov_control(
penalty_structure = "GROUP_CENTERING", prior_groups = groups,
lambda_2 = 1),
control = ctrl)
gc$select()
gc$selected_indices
#> [1] 21 22 23 24 25 26 27 28 29 30The selector runs "TIKNET", the pathwise homotopy
algorithm of the Tikhonov net. It is the solver of the base control that
applies when control is left out.
trex_control() names "TLARS" by default, which
the selector refuses as every other solver, so a base control that is
passed sets solver = "TIKNET".