Developer Guide and Reference

  • 2021.4
  • 09/27/2021
  • Public Content
Contents

AdaBoost Multiclass Classifier

AdaBoost (short for “Adaptive Boosting”) is a popular boosting classification algorithm. The AdaBoost algorithm performs well on a variety of data sets except some noisy data ([Friedman98], [Zhu2005]). The library supports two methods for the algorithms:
  • SAMME, or Stagewise Additive Modeling using a Multi-class Exponential loss function [Zhu2005]
  • SAMME.R, which is a modification of SAMME method for Real-valued returned probabilities from weak learner

Details

Given
n
feature vectors LaTex Math image. of size
p
, a vector of class labels LaTex Math image. where LaTex Math image. in case of binary classification and LaTex Math image., where
C
is a number of classes, describes the class
t
the feature vector LaTex Math image. belongs to, and LaTex Math image. is a weak learner algorithm, the problem is to build an AdaBoost classifier.

Training Stage

SAMME method
The following scheme shows the major steps of the SAMME algorithm:
  1. Initialize weights LaTex Math image. for LaTex Math image.
  2. For LaTex Math image.:
    • Train the weak learner LaTex Math image. using weights LaTex Math image..
    • Choose a confidence value LaTex Math image., where LaTex Math image.
    • Update LaTex Math image., where LaTex Math image. is a normalization factor.
  3. Output the final hypothesis:
    LaTex Math image.
SAMME algorithm in case of binary classification is equal to the AdaBoost algorithm from [Friedman98].
SAMME.R method
The following scheme shows the major steps of the SAMME.R algorithm:
  1. Initialize weights LaTex Math image. for LaTex Math image.
  2. For LaTex Math image.:
    • Train the weak learner LaTex Math image. using weights LaTex Math image..
    • Receive the weighed class probability estimates from weak learner:
      LaTex Math image.
    • For LaTex Math image., set LaTex Math image.:
      LaTex Math image.
    • For LaTex Math image., update LaTex Math image.:
      LaTex Math image.
      where LaTex Math image. is a normalization factor, LaTex Math image., LaTex Math image.
  3. Output the final hypothesis:
    LaTex Math image.
Prediction Stage
SAMME method
Given the AdaBoost classifier and
r
feature vectors LaTex Math image., the problem is to calculate the final class
H(x)
:
LaTex Math image.
SAMME.R method
Given the AdaBoost classifier and
r
feature vectors LaTex Math image., the problem is to calculate the final class
H(x)
:
LaTex Math image.
where LaTex Math image. is as defined above in Training Stage.

Batch Processing

AdaBoost classifier follows the general workflow described in Classification Usage Model.
Training
For a description of the input and output, refer to Classification Usage Model. At the training stage, an AdaBoost classifier has the following parameters:
Parameter
Default Value
Description
algorithmFPType
float
The floating-point type that the algorithm uses for intermediate computations. Can be
float
or
double
.
method
defaultDense
Available methods for computation of the AdaBoost algorithm:
  • samme
    - uses the classifier that returns labels as weak learner
  • sammeR
    - uses the classifier that returns probabilities of belonging to class as weak learner
  • defaultDense
    is equal to
    samme
    method
weakLearnerTraining
Pointer to an object of the classification stump training class
Pointer to the training algorithm of the weak learner. By default, a classification stump weak learner is used.
weakLearnerPrediction
Pointer to an object of the classification stump prediction class
Pointer to the prediction algorithm of the weak learner. By default, a classification stump weak learner is used.
accuracyThreshold
0.01
AdaBoost training accuracy.
maxIterations
100
The maximal number of iterations for the algorithm.
learningRate
1.0
Multiplier for each classifier to shrink its contribution.
nClasses
2
The number of classes.
resultsToCompute
0
The 64-bit integer flag that specifies which extra characteristics of AdaBoost to compute. Current version of the library only provides the following option:
computeWeakLearnersErrors
Output
In addition to classifier output, AdaBoostcalculates the result described below. Pass the
Result ID
as a parameter to the methods that access the result of your algorithm. For more details, see Algorithms.
Result ID
Result
weakLearnersErrors
A numeric table LaTex Math image. containing weak learner’s classification errors computed when the
computeWeakLearnersErrors
option is on.
By default, this result is an object of the
HomogenNumericTable
class, but you can define the result as an object of any class derived from
NumericTable
.
Prediction
For a description of the input and output, refer to Classification Usage Model. At the prediction stage, an AdaBoost classifier has the following parameters:
Parameter
Default Value
Description
algorithmFPType
float
The floating-point type that the algorithm uses for intermediate computations. Can be
float
or
double
.
method
defaultDense
Performance-oriented computation method, the only method supported by the AdaBoost classifier at the prediction stage.
weakLearnerPrediction
Pointer to an object of the classification stump prediction class
Pointer to the prediction algorithm of the weak learner. By default, a classification stump weak learner is used.
nClasses
2
The number of classes.

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.