Developer Guide and Reference

  • 2021.4
  • 09/27/2021
  • Public Content
Contents

LogitBoost Classifier

LogitBoost is a boosting classification algorithm. LogitBoost and AdaBoost are close to each other in the sense that both perform an additive logistic regression. The difference is that AdaBoost minimizes the exponential loss, whereas LogitBoost minimizes the logistic loss.
LogitBoost within oneDAL implements a multi-class classifier.

Details

Given
n
feature vectors LaTex Math image. of size
p
and a vector of class labels LaTex Math image., where LaTex Math image. describes the class to which the feature vector LaTex Math image. belongs and
J
is the number of classes, the problem is to build a multi-class LogitBoost classifier.
Training Stage
The LogitBoost model is trained using the Friedman method [Friedman00].
Let LaTex Math image. is the indicator that the
i
-th feature vector belongs to class
j
. The scheme below, which uses the stump weak learner, shows the major steps of the algorithm:
  1. Start with weights LaTex Math image., LaTex Math image., LaTex Math image., LaTex Math image., LaTex Math image..
  2. For LaTex Math image.:
    Do
    For LaTex Math image.
    Do
    1. Compute working responses and weights in the j-th class:
      LaTex Math image.
      LaTex Math image.
    2. Fit the function LaTex Math image. by a weighted least-squares regression of LaTex Math image. to LaTex Math image. with weights LaTex Math image. using the stump-based approach.
    End do
    LaTex Math image.
    LaTex Math image.
    LaTex Math image.
    End do
The result of the model training is a set of
M
stumps.
Prediction Stage
Given the LogitBoost classifier and
r
feature vectors LaTex Math image., the problem is to calculate the labels LaTex Math image. of the classes to which the feature vectors belong.

Batch Processing

LogitBoost classifier follows the general workflow described in Classification Usage Model.
Training
For a description of the input and output, refer to Classification Usage Model.
At the training stage, a LogitBoost classifier has the following parameters:
Parameter
Default Value
Description
algorithmFPType
float
The floating-point type that the algorithm uses for intermediate computations. Can be
float
or
double
.
method
defaultDense
The computation method used by the LogitBoost classifier. The only training method supported so far is the Friedman method.
weakLearnerTraining
DEPRECATED
: Pointer to an object of the stump training class.
USE INSTEAD
: Pointer to an object of the regression stump training class.
DEPRECATED
: Pointer to the training algorithm of the weak learner. By default, a stump weak learner is used.
USE INSTEAD
: Pointer to the regression training algorithm. By default, a regression stump with mse split criterion is used.
weakLearnerPrediction
DEPRECATED
: Pointer to an object of the stump prediction class.
USE INSTEAD
: Pointer to an object of the regression stump prediction class.
DEPRECATED
: Pointer to the prediction algorithm of the weak learner. By default, a stump weak learner is used.
USE INSTEAD
: Pointer to the regression prediction algorithm. By default, a regression stump with mse split criterion is used.
accuracyThreshold
0.01
LogitBoost training accuracy.
maxIterations
100
The maximal number of iterations for the LogitBoost algorithm.
nClasses
Not applicable
The number of classes, a required parameter.
weightsDegenerateCasesThreshold
LaTex Math image.
The threshold to avoid degenerate cases when calculating weights LaTex Math image..
responsesDegenerateCasesThreshold
LaTex Math image.
The threshold to avoid degenerate cases when calculating responses LaTex Math image..
Prediction
For a description of the input and output, refer to Classification Usage Model.
At the prediction stage, a LogitBoost classifier has the following parameters:
Parameter
Default Value
Description
algorithmFPType
float
The floating-point type that the algorithm uses for intermediate computations. Can be
float
or
double
.
method
defaultDense
Performance-oriented computation method, the only method supported by the LogitBoost classifier at the prediction stage.
weakLearnerPrediction
DEPRECATED
: Pointer to an object of the stump prediction class.
USE INSTEAD
: Pointer to an object of the regression stump prediction class.
DEPRECATED
: Pointer to the prediction algorithm of the weak learner. By default, a stump weak learner is used.
USE INSTEAD
: Pointer to the regression prediction algorithm. By default, a regression stump with mse split criterion is used.
nClasses
Not applicable
The number of classes, a required parameter.
The algorithm terminates if it achieves the specified accuracy or reaches the specified maximal number of iterations. To determine the actual number of iterations performed, call the
getNumberOfWeakLearners()
method of the
LogitBoostModel
class and divide it by
nClasses
.

Examples

C++ (CPU)
Batch Processing:
Java*
There is no support for Java on GPU.
Batch Processing:
Python*
Batch Processing:

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.