Developer Guide and Reference

  • 2021.4
  • 09/27/2021
  • Public Content
Contents

Quality Metrics for Multi-class Classification Algorithms

For
l
classes LaTex Math image., given a vector LaTex Math image. of class labels computed at the prediction stage of the classification algorithm and a vector LaTex Math image. of expected class labels, the problem is to evaluate the classifier by computing the confusion matrix and connected quality metrics: precision, error rate, and so on.
QualityMetricsId
for multi-class classification is
confusionMatrix
.

Details

Further definitions use the following notations:
LaTex Math image.
true positive
the number of correctly recognized observations for class LaTex Math image.
LaTex Math image.
true negative
the number of correctly recognized observations that do not belong to the class LaTex Math image.
LaTex Math image.
false positive
the number of observations that were incorrectly assigned to the class LaTex Math image.
ERROR processing math
false negative
the number of observations that were not recognized as belonging to the class LaTex Math image.
The library uses the following quality metrics for multi-class classifiers:
Quality Metric
Definition
Average accuracy
LaTex Math image.
Error rate
LaTex Math image.
Micro precision (LaTex Math image.)
LaTex Math image.
Micro recall (LaTex Math image.)
LaTex Math image.
Micro F-score (LaTex Math image.)
LaTex Math image.
Macro precision (LaTex Math image.)
LaTex Math image.
Macro recall (LaTex Math image.)
LaTex Math image.
Macro F-score (LaTex Math image.)
LaTex Math image.
For more details of these metrics, including the evaluation focus, refer to [Sokolova09].
The following is the confusion matrix:
Classified as Class LaTex Math image.
LaTex Math image.
Classified as Class LaTex Math image.
LaTex Math image.
Classified as Class LaTex Math image.
Actual Class LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.
Actual Class LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.
Actual Class LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.
The positives and negatives are defined through elements of the confusion matrix as follows:
LaTex Math image.
LaTex Math image.
LaTex Math image.
LaTex Math image.

Batch Processing

Algorithm Input
The quality metric algorithm for multi-class classifiers accepts the input described below. Pass the
Input ID
as a parameter to the methods that provide input for your algorithm. For more details, see Algorithms.
Input ID
Input
predictedLabels
Pointer to the LaTex Math image. numeric table that contains labels computed at the prediction stage of the classification algorithm.
This input can be an object of any class derived from
NumericTable
except
PackedSymmetricMatrix
,
PackedTriangularMatrix
, and
CSRNumericTable
.
groundTruthLabels
Pointer to the LaTex Math image. numeric table that contains expected labels.
This input can be an object of any class derived from NumericTable except
PackedSymmetricMatrix
,
PackedTriangularMatrix
, and
CSRNumericTable
.
Algorithm Parameters
The quality metric algorithm has the following parameters:
Parameter
Default Value
Description
algorithmFPType
float
The floating-point type that the algorithm uses for intermediate computations. Can be
float
or
double
.
method
defaultDense
Performance-oriented computation method, the only method supported by the algorithm.
nClasses
0
The number of classes (
l
).
useDefaultMetrics
true
A flag that defines a need to compute the default metrics provided by the library.
beta
1
The LaTex Math image. parameter of the F-score quality metric provided by the library.
Algorithm Output
The quality metric algorithm calculates the result described below. Pass the
Result ID
as a parameter to the methods that access the results of your algorithm. For more details, see Algorithms.
Result ID
Result
confusionMatrix
Pointer to the LaTex Math image. numeric table with the confusion matrix.
By default, this result is an object of the
HomogenNumericTable
class, but you can define the result as an object of any class derived from NumericTable except
PackedTriangularMatrix
,
PackedSymmetricMatrix
, and
CSRNumericTable
.
multiClassMetrics
Pointer to the LaTex Math image. numeric table that contains quality metrics, which you can access by an appropriate Multi-class Metrics ID:
  • averageAccuracy
    - average accuracy
  • errorRate
    - error rate
  • microPrecision
    - micro precision
  • microRecall
    - micro recall
  • microFscore
    - micro F-score
  • macroPrecision
    - macro precision
  • macroRecall
    - macro recall
  • macroFscore
    - macro F-score
By default, this result is an object of the
HomogenNumericTable
class, but you can define the result as an object of any class derived from NumericTable except
PackedTriangularMatrix
,
PackedSymmetricMatrix
, and
CSRNumericTable
.

Examples

C++ (CPU)
Batch Processing:
Java*
There is no support for Java on GPU.
Batch Processing:

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.