Coordinate Descent Algorithm
The Coordinate Descent algorithm follows the algorithmic framework of iterative solver with one exception:
the default method (
defaultDense
) of Coordinate Descent algorithm is a case of the iterative solver method
with the batch equal to the number of observations in the training data set.Details
The aet of intrinsic parameters 
is empty.
Algorithmic-specific transformation 
, algorithm-specific vector 
,
and power 
of Lebesgue space [Adams2003] are defined as follows:
is empty.
Algorithmic-specific transformation , algorithm-specific vector ,
and power of Lebesgue space [Adams2003] are defined as follows:
- Define the index
to update the component of a coefficient as a remainder in the division of the number of current iteration (
)
by the number of features in the training data set (
): 
Alternatively, ifselectionparameter was set torandom, generate randomly. - IfstepLengthSequencewas not provided by the user, compute the learning rate:
(the diagonal element of the Hessian matrix) - Update the-th component of vector
:
Note: for example, if a non-smooth term
,
where is the number of features in the training data set, the objective function should compute prox operator as follows:
Convergence check is performed each 
iterations:
iterations:
, 
- For
, the infinity norm () is defined as follows:
Computation
Coordinate Descent algorithm is a special case of an iterative solver.
For parameters, input, and output of iterative solvers, see Iterative Solver > Computation.
Algorithm Parameters
In addition to the input of a iterative solver, Coordinate Descent algorithm accepts the following parameters:
Parameter | Default Value | Description |
|---|---|---|
algorithmFPType | float | The floating-point type that the algorithm uses for intermediate computations. Can be float or double . |
method | defaultDense | Performance-oriented method. |
engine | SharePtr< engines:: mt19937:: Batch>() | Pointer to the random number generator engine that is used internally during each iteration
to choose a random component of the minimum result vector to be updated. |
positive | false | A boolean value. When set to true , it forces the coefficients to be positive. |
selection | cyclic | Value that specifies the strategy of certain coordinate selection on each iteration.
Except for default cyclic value, Coordinate Descent also supports:
|
skipTheFirstComponents | false | A boolean value. When set to true , Coordinate Descent algorithm will skip the first component from optimization. |