Intel® oneAPI Data Analytics Library Developer Guide and Reference
Developer Guide and Reference
Elastic Net is a method for modeling relationship between a dependent variable (which may be a vector)
and one or more explanatory variables by fitting regularized least squares model.
Elastic Net regression model has the special penalty, a sum of L1 and L2 regularizations,
that takes advantage of both Ridge Regression and LASSO algorithms.
This penalty is particularly useful in a situation with many correlated predictor variables [Friedman2010].
Let be a vector of input variables and be the response.
For each , the Elastic Net model has the form similar to linear and ridge regression models [Hoerl70]
with one exception: the coefficients are estimated by minimizing mean squared error (MSE) objective function that is
regularized by and penalties.
Here , , are referred to as independent variables,
, , is referred to as dependent variable or response.
Let be a set of
training data (for regression task, , and for feature selection could be greater than ).
The matrix of size contains observations , ,
of independent variables.
For each , , the Elastic Net regression estimates
by minimizing the objective function:
In the equation above, the first term is a mean squared error function, the second and the third
are regularization terms that penalize the and norms of vector ,
where , , .