Quality Metrics for Binary Classification Algorithms

Intel® oneAPI Data Analytics Library Developer Guide and Reference

Download PDF

ID 772611

Date 7/13/2023

Version

Public

A newer version of this document is available. Customers should click here to go to the newest version.

Visible to Intel only — GUID: GUID-2485D6E0-6E28-49B8-B27A-57B18C280AD7

View Details

Quality Metrics for Binary Classification Algorithms

For two classes $C_{1}$ and $C 2$ , given a vector $X = (x_{1}, \dots, x_{n})$ of class labels computed at the prediction stage of the classification algorithm and a vector $Y = (y_{1}, \dots, y_{n})$ of expected class labels, the problem is to evaluate the classifier by computing the confusion matrix and connected quality metrics: precision, recall, and so on.

QualityMetricsId for binary classification is confusionMatrix.

Details

Further definitions use the following notations:

Notations for Quality Metrics for Binary Classification Algorithms
$tp$	true positive	the number of correctly recognized observations for class $C_{1}$
$tn$	true negative	the number of correctly recognized observations that do not belong to the class $C_{1}$
$fp$	false positive	the number of observations that were incorrectly assigned to the class $C_{1}$
$fn$	false negative	the number of observations that were not recognized as belonging to the class $C_{1}$

The library uses the following quality metrics for binary classifiers:

Definitions of Quality Metrics for Binary Classification Algorithms
Quality Metric	Definition
Accuracy	$\frac{tp + tn}{tp + fn + fp + tn}$
Precision	$\frac{tp}{tp + fp}$
Recall	$\frac{tp}{tp + fn}$
F-score	$\frac{(β^{2} + 1) tp}{(β^{2} + 1) tp + β^{2} fn + fp}$
Specificity	$\frac{tn}{fp + tn}$
Area under curve (AUC)	$\frac{1}{2} (\frac{tp}{tp + fn} + \frac{tn}{tn + fp})$

For more details of these metrics, including the evaluation focus, refer to [Sokolova09].

The confusion matrix is defined as follows:

Confusion Matrix for Binary Classification Algorithms
	Classified as Class $C_{1}$	Classified as Class $C_{2}$
Actual Class $C_{1}$	tp	fn
Actual Class $C_{2}$	fp	tn

Batch Processing

Algorithm Input

The quality metric algorithm for binary classifiers accepts the input described below. Pass the Input ID as a parameter to the methods that provide input for your algorithm. For more details, see Algorithms.

Algorithm Input for Quality Metrics for Binary Classification (Batch Processing)
Input ID	Input
`predictedLabels`	Pointer to the $n i m e s 1$ numeric table that contains labels computed at the prediction stage of the classification algorithm. This input can be an object of any class derived from `NumericTable` except `PackedSymmetricMatrix`, `PackedTriangularMatrix`, and `CSRNumericTable`.
`groundTruthLabels`	Pointer to the $n i m e s 1$ numeric table that contains expected labels. This input can be an object of any class derived from `NumericTable` except `PackedSymmetricMatrix`, `PackedTriangularMatrix`, and `CSRNumericTable`.

Algorithm Parameters

The quality metric algorithm has the following parameters:

Algorithm Parameters for Quality Metrics for Binary Classification (Batch Processing)
Parameter	Default Value	Description
`algorithmFPType`	`float`	The floating-point type that the algorithm uses for intermediate computations. Can be `float` or `double`.
`method`	`defaultDense`	Performance-oriented computation method, the only method supported by the algorithm.
`beta`	1	The $β$ parameter of the F-score quality metric provided by the library.

Algorithm Output

The quality metric algorithm calculates the result described below. Pass the Result ID as a parameter to the methods that access the results of your algorithm. For more details, see Algorithms.

Algorithm Output for Quality Metrics for Binary Classification (Batch Processing)
Result ID	Result
`confusionMatrix`	Pointer to the $2 \times 2$ numeric table with the confusion matrix. NOTE: By default, this result is an object of the `HomogenNumericTable` class, but you can define the result as an object of any class derived from NumericTable except `PackedTriangularMatrix`, `PackedSymmetricMatrix`, and `CSRNumericTable`.
`binaryMetrics`	Pointer to the $1 \times 6$ numeric table that contains quality metrics, which you can access by an appropriate Binary Metrics ID: `accuracy` - accuracy `precision` - precision `recall` - recall `fscore` - F-score `specificity` - specificity `AUC` - area under the curve NOTE: By default, this result is an object of the `HomogenNumericTable` class, but you can define the result as an object of any class derived from `NumericTable` except `PackedTriangularMatrix`, `PackedSymmetricMatrix`, and `CSRNumericTable`.

Examples

C++ (CPU)

Batch Processing:

svm_two_class_metrics_dense_batch.cpp

Java*

NOTE:

The Java interfaces are deprecated in the oneDAL library and may no longer be supported in future releases.

Batch Processing:

SVMTwoClassMetricsDenseBatch.java

Select Your Language

Using Intel.com Search

Quick Links

Recent Searches

Advanced Search

Only search in

Intel® oneAPI Data Analytics Library Developer Guide and Reference

Quality Metrics for Binary Classification Algorithms

Details

Batch Processing

Examples