Coordinate Descent Algorithm
The Coordinate Descent algorithm follows the algorithmic framework of iterative solver with one exception:
the default method (
defaultDense
) of Coordinate Descent algorithm is a case of the iterative solver method
with the batch equal to the number of observations in the training data set.Details
The aet of intrinsic parameters
is empty.
Algorithmic-specific transformation
, algorithm-specific vector
,
and power
of Lebesgue space [Adams2003] are defined as follows:
- Define the index
to update the component of a coefficient as a remainder in the division of the number of current iteration (
) by the number of features in the training data set (
):
Alternatively, ifselectionparameter was set torandom, generaterandomly.
- IfstepLengthSequencewas not provided by the user, compute the learning rate:
(the diagonal element of the Hessian matrix)
- Update the
-th component of vector
:
Note: for example, if a non-smooth term, where
is the number of features in the training data set, the objective function should compute prox operator as follows:
Convergence check is performed each
iterations:
,
- For
, the infinity norm (
) is defined as follows:
Computation
Coordinate Descent algorithm is a special case of an iterative solver.
For parameters, input, and output of iterative solvers, see Iterative Solver > Computation.
Algorithm Parameters
In addition to the input of a iterative solver, Coordinate Descent algorithm accepts the following parameters:
Parameter | Default Value | Description |
---|---|---|
algorithmFPType | float | The floating-point type that the algorithm uses for intermediate computations. Can be float or double . |
method | defaultDense | Performance-oriented method. |
engine | SharePtr< engines:: mt19937:: Batch>() | Pointer to the random number generator engine that is used internally during each iteration
to choose a random component of the minimum result vector to be updated. |
positive | false | A boolean value. When set to true , it forces the coefficients to be positive. |
selection | cyclic | Value that specifies the strategy of certain coordinate selection on each iteration.
Except for default cyclic value, Coordinate Descent also supports:
|
skipTheFirstComponents | false | A boolean value. When set to true , Coordinate Descent algorithm will skip the first component from optimization. |