Fit Ridge Regression Across a Grid of Penalty Values

Fits ridge regression (adapted from MASS::lm.ridge) of y on x for one or several ridge penalty (lambda) values simultaneously, using an SVD-based solution path. Predictors are centered and scaled internally (columns with zero or undefined standard deviation are left unscaled), and both generalized cross-validation (GCV) and BIC are computed for each lambda to help with penalty selection.

ridge(x, y, lambda = 10^seq(-10, 10, length.out = 100))

Arguments

x

A numeric matrix (or vector/data frame coercible to a matrix) of predictors, with observations in rows and variables in columns.

y

A numeric response vector, of the same length as nrow(x).

lambda

A numeric vector of ridge penalty values to fit. Defaults to a grid of 100 values log-spaced between 1e-10 and 1e10.

Value

An object of class "ridge", a list with components:

coef

Ridge regression coefficients; a vector if a single lambda was supplied, otherwise a matrix with one column per lambda.

ym

Mean of the response y, used for centering.

xm

Column means of x, used for centering.

xsd

Column standard deviations of x used for scaling (zero/NA values replaced by 1).

lambda

The lambda grid used.

best_lam

The value of lambda minimizing the GCV score.

fitted_values

Fitted values on the original scale of y; a vector or a matrix with one column per lambda.

residuals

Residuals (centered y minus centered fitted values); a vector or matrix matching fitted_values.

GCV

Generalized cross-validation score for each lambda.

BIC

Bayesian information criterion for each lambda.

x

The (unscaled) predictor matrix used for fitting.

y

The original response vector.

Details

The predictor matrix x is coerced to a matrix (as a single column if it has only one column or one row) and centered/scaled by column means and standard deviations; columns with zero or NA standard deviation are assigned a scale of 1 to avoid division by zero. The response y is mean-centered. Ridge coefficients for the full lambda grid are then computed in one pass via the singular value decomposition of the scaled, centered design matrix (La.svd), following the same approach as MASS::lm.ridge.

For each value of lambda, the function also computes:

  • fitted values (on the original scale of y);

  • residuals;

  • the generalized cross-validation (GCV) score, used to select best_lam as the lambda minimizing GCV;

  • the Bayesian information criterion (BIC).

When length(lambda) > 1, coef and fitted_values are returned as matrices with one column per lambda (column names set to the corresponding lambda values); when a single lambda is supplied, they are returned as vectors.

Examples

if (FALSE) { # \dontrun{
set.seed(1)
x <- matrix(rnorm(100 * 5), ncol = 5)
y <- x %*% c(1, 0, -1, 0, 2) + rnorm(100)
fit <- ridge(x, y)
fit$best_lam
plot(fit$lambda, fit$GCV, log = "x", type = "l")
} # }