Developer Guide and Reference

  • 2021.4
  • 09/27/2021
  • Public Content
Contents

Decision Forest Classification and Regression (DF)

Decision Forest (DF) classification and regression algorithms are based on an ensemble of tree-structured classifiers, which are known as decision trees. Decision forest is built using the general technique of bagging, a bootstrap aggregation, and a random choice of features. For more details, see [Breiman84] and [Breiman2001].

Programming Interface

All types and functions in this section are declared in the
oneapi::dal::decision_forest
namespace and are available via inclusion of the
oneapi/dal/algo/decision_forest.hpp
header file.
Enum classes
enum class
error_metric_mode
error_metric_mode::none
Do not compute error metric.
error_metric_mode::out_of_bag_error
Train produces LaTex Math image. table with cumulative prediction error for out of bag observations.
error_metric_mode::out_of_bag_error_per_observation
Train produces LaTex Math image. table with prediction error for out-of-bag observations.
enum class
variable_importance_mode
variable_importance_mode::none
Do not compute variable importance.
variable_importance_mode::mdi
Mean Decrease Impurity. Computed as the sum of weighted impurity decreases for all nodes where the variable is used, averaged over all trees in the forest.
variable_importance_mode::mda_raw
Mean Decrease Accuracy (permutation importance). For each tree, the prediction error on the out-of-bag portion of the data is computed (error rate for classification, MSE for regression). The same is done after permuting each predictor variable. The difference between the two are then averaged over all trees.
variable_importance_mode::mda_scaled
Mean Decrease Accuracy (permutation importance). This is MDA_Raw value scaled by its standard deviation.
enum class
infer_mode
infer_mode::class_labels
Infer produces a LaTex Math image. table with the predicted labels.
infer_mode::class_responses
deprecated.
infer_mode::class_probabilities
Infer produces LaTex Math image. table with the predicted class probabilities for each observation.
enum class
voting_mode
voting_mode::weighted
The final prediction is combined through a weighted majority voting.
voting_mode::unweighted
The final prediction is combined through a simple majority voting.
Descriptor
template<typename
Float
= float, typename
Method
= method::by_default, typename
Task
= task::by_default>
class
descriptor
Template Parameters
  • Float
    – The floating-point type that the algorithm uses for intermediate computations. Can be
    float
    or
    double
    .
  • Method
    – Tag-type that specifies an implementation of algorithm. Can be
    method::dense
    or
    method::hist
    .
  • Task
    – Tag-type that specifies type of the problem to solve. Can be or .
Constructors
descriptor
() = default
Creates a new instance of the class with the default property values.
Properties
std::int64_t
min_observations_in_leaf_node
The minimal number of observations in a leaf node.
Default value
: 1 for classification, 5 for regression.
Getter & Setter


std::int64_t get_min_observations_in_leaf_node() const
auto & set_min_observations_in_leaf_node(std::int64_t value)

Invariants


min_observations_in_leaf_node > 0

variable_importance_mode
variable_importance_mode
The variable importance mode.
Default value
: variable_importance_mode::none.
Getter & Setter


variable_importance_mode get_variable_importance_mode() const
auto & set_variable_importance_mode(variable_importance_mode value)

std::int64_t
class_count
The class count. Used with only.
Default value
: 2.
Getter & Setter


template > std::int64_t get_class_count() const
template > auto & set_class_count(std::int64_t value)

std::int64_t
max_leaf_nodes
The maximal number of the leaf nodes. If 0, the number of leaf nodes is not limited.
Default value
: 0.
Getter & Setter


std::int64_t get_max_leaf_nodes() const
auto & set_max_leaf_nodes(std::int64_t value)

bool
bootstrap
The bootstrap mode, if true, the training set for a tree is a bootstrap of the whole training set, if False, the whole dataset is used to build each tree.
Default value
: true.
Getter & Setter


bool get_bootstrap() const
auto & set_bootstrap(bool value)

std::int64_t
features_per_node
The number of features to consider when looking for the best split for a node.
Default value
: task::classification ? sqrt(p) : p/3, where p is the total number of features.
Getter & Setter


std::int64_t get_features_per_node() const
auto & set_features_per_node(std::int64_t value)

error_metric_mode
error_metric_mode
The error metric mode.
Default value
: error_metric_mode::none.
Getter & Setter


error_metric_mode get_error_metric_mode() const
auto & set_error_metric_mode(error_metric_mode value)

double
min_weight_fraction_in_leaf_node
The min weight fraction in a leaf node. The minimum weighted fraction of the total sum of weights (of all input observations) required to be at a leaf node.
Default value
: 0.0.
Getter & Setter


double get_min_weight_fraction_in_leaf_node() const
auto & set_min_weight_fraction_in_leaf_node(double value)

Invariants


min_weight_fraction_in_leaf_node >= 0.0
min_weight_fraction_in_leaf_node <= 0.5

std::int64_t
min_bin_size
The minimal number of observations in a bin. Used with
method::hist
split-finding method only.
Default value
: 5.
Getter & Setter


std::int64_t get_min_bin_size() const
auto & set_min_bin_size(std::int64_t value)

Invariants


min_bin_size > 0

std::int64_t
max_bins
The maximal number of discrete bins to bucket continuous features. Used with
method::hist
split-finding method only. Increasing the number results in higher computation costs.
Default value
: 256.
Getter & Setter


std::int64_t get_max_bins() const
auto & set_max_bins(std::int64_t value)

Invariants


max_bins > 1

std::int64_t
max_tree_depth
The maximal depth of the tree. If 0, then nodes are expanded until all leaves are pure or until all leaves contain less or equal to min observations in leaf node samples.
Default value
: 0.
Getter & Setter


std::int64_t get_max_tree_depth() const
auto & set_max_tree_depth(std::int64_t value)

std::int64_t
min_observations_in_split_node
The minimal number of observations in a split node.
Default value
: 2.
Getter & Setter


std::int64_t get_min_observations_in_split_node() const
auto & set_min_observations_in_split_node(std::int64_t value)

Invariants


min_observations_in_split_node > 1

std::int64_t
tree_count
The number of trees in the forest.
Default value
: 100.
Getter & Setter


std::int64_t get_tree_count() const
auto & set_tree_count(std::int64_t value)

Invariants


tree_count > 0

infer_mode
infer_mode
The infer mode. Used with only.
Getter & Setter


template > infer_mode get_infer_mode() const
template > auto & set_infer_mode(infer_mode value)

voting_mode
voting_mode
The voting mode. Used with only.
Getter & Setter


template > voting_mode get_voting_mode() const
template > auto & set_voting_mode(voting_mode value)

double
min_impurity_decrease_in_split_node
The min impurity decrease in a split node is a threshold for stopping the tree growth early. A node will be split if its impurity is above the threshold, otherwise it is a leaf.
Default value
: 0.0.
Getter & Setter


double get_min_impurity_decrease_in_split_node() const
auto & set_min_impurity_decrease_in_split_node(double value)

Invariants


min_impurity_decrease_in_split_node >= 0.0

bool
memory_saving_mode
The memory saving mode.
Default value
: false.
Getter & Setter


bool get_memory_saving_mode() const
auto & set_memory_saving_mode(bool value)

double
observations_per_tree_fraction
The fraction of observations per tree.
Default value
: 1.0.
Getter & Setter


double get_observations_per_tree_fraction() const
auto & set_observations_per_tree_fraction(double value)

Invariants


observations_per_tree_fraction > 0.0
observations_per_tree_fraction <= 1.0

double
impurity_threshold
The impurity threshold, a node will be split if this split induces a decrease of the impurity greater than or equal to the input value.
Default value
: 0.0.
Getter & Setter


double get_impurity_threshold() const
auto & set_impurity_threshold(double value)

Invariants


impurity_threshold >= 0.0

Method tags
struct
dense
Tag-type that denotes dense computational method.
struct
hist
Tag-type that denotes hist computational method.
using
by_default
= dense
Alias tag-type for dense computational method.
Task tags
struct
classification
Tag-type that parameterizes entities used for solving classification problem.
struct
regression
Tag-type that parameterizes entities used for solving regression problem.
using
by_default
= classification
Alias tag-type for classification task.
Model
template<typename
Task
= task::by_default>
class
model
Template Parameters
Task
– Tag-type that specifies the type of the problem to solve. Can be or .
Constructors
model
()
Creates a new instance of the class with the default property values.
Public Methods
std::int64_t
get_tree_count
()
const
The number of trees in the forest.
template<typename
T
= Task, typename
None
= detail::enable_if_classification_t<T>> std::int64_t
get_class_count
()
const
The class count. Used with
oneapi::dal::decision_forest::task::classification
only.
template<typename
Visitor
> void
traverse_depth_first
(std::int64_t
tree_idx
, Visitor &&
visitor
)
const
Performs Depth First Traversal of i-th tree.
Parameters
  • tree_idx
    – Index of the tree to traverse.
  • visitor
    – This functor gets notified when tree nodes are visited, via corresponding operators: bool operator()(const decision_forest::split_node_info<Task>&) bool operator()(const decision_forest::leaf_node_info<Task>&).
template<typename
Visitor
> void
traverse_breadth_first
(std::int64_t
tree_idx
, Visitor &&
visitor
)
const
Performs Breadth First Traversal of i-th tree.
Parameters
  • tree_idx
    – Index of the tree to traverse.
  • visitor
    – This functor gets notified when tree nodes are visited, via corresponding operators: bool operator()(const decision_forest::split_node_info<Task>&) bool operator()(const decision_forest::leaf_node_info<Task>&).
Training
train(...)
Input
template<typename
Task
= task::by_default>
class
train_input
Template Parameters
Task
– Tag-type that specifies type of the problem to solve. Can be or .
Constructors
train_input
(
const
table &
data
,
const
table &
responses
)
Creates a new instance of the class with the given
data
and
responses
property values.
Properties
const
table &
responses
Vector of responses LaTex Math image. for the training set LaTex Math image..
Default value
: table{}.
Getter & Setter


const table & get_responses() const
auto & set_responses(const table &value)

const
table &
labels
Vector of labels LaTex Math image. for the training set LaTex Math image..
Default value
: table{}.
Getter & Setter


const table & get_labels() const
auto & set_labels(const table &value)

const
table &
data
The training set LaTex Math image..
Default value
: table{}.
Getter & Setter


const table & get_data() const
auto & set_data(const table &value)

Result
template<typename
Task
= task::by_default>
class
train_result
Template Parameters
Task
– Tag-type that specifies type of the problem to solve. Can be or .
Constructors
train_result
()
Creates a new instance of the class with the default property values.
Properties
const
table &
oob_err_per_observation
A LaTex Math image. table containing out-of-bag error value per observation. Computed when
error_metric_mode
set with
error_metric_mode::out_of_bag_error_per_observation
.
Default value
: table{}.
Getter & Setter


const table & get_oob_err_per_observation() const
auto & set_oob_err_per_observation(const table &value)

const
table &
var_importance
A LaTex Math image. table containing variable importance value for each feature. Computed when .
Default value
: table{}.
Getter & Setter


const table & get_var_importance() const
auto & set_var_importance(const table &value)

const
model<Task> &
model
The trained Decision Forest model.
Default value
: model<Task>{}.
Getter & Setter


const model< Task > & get_model() const
auto & set_model(const model< Task > &value)

const
table &
oob_err
A LaTex Math image. table containing cumulative out-of-bag error value. Computed when
error_metric_mode
set with
error_metric_mode::out_of_bag_error
.
Default value
: table{}.
Getter & Setter


const table & get_oob_err() const
auto & set_oob_err(const table &value)

Operation
template<typename
Descriptor
> decision_forest::train_result
train
(
const
Descriptor &
desc
,
const
decision_forest::train_input &
input
)
Parameters
  • desc
    – Decision Forest algorithm descriptor
    decision_forest::descriptor
    .
  • input
    – Input data for the training operation
Preconditions


input.data.is_empty == false
input.labels.is_empty == false
input.labels.column_count == 1
input.data.row_count == input.labels.row_count
desc.get_bootstrap() == true || (desc.get_bootstrap() == false && desc.get_variable_importance_mode() != variable_importance_mode::mda_raw && desc.get_variable_importance_mode() != variable_importance_mode::mda_scaled)
desc.get_bootstrap() == true || (desc.get_bootstrap() == false && desc.get_error_metric_mode() == error_metric_mode::none)

Inference
infer(...)
Input
template<typename
Task
= task::by_default>
class
infer_input
Template Parameters
Task
– Tag-type that specifies the type of the problem to solve. Can be or .
Constructors
infer_input
(
const
model<Task> &
trained_model
,
const
table &
data
)
Creates a new instance of the class with the given
model
and
data
property values.
Properties
const
model<Task> &
model
The trained Decision Forest model.
Default value
: model<Task>{}.
Getter & Setter


const model< Task > & get_model() const
auto & set_model(const model< Task > &value)

const
table &
data
The dataset for inference LaTex Math image..
Default value
: table{}.
Getter & Setter


const table & get_data() const
auto & set_data(const table &value)

Result
template<typename
Task
= task::by_default>
class
infer_result
Template Parameters
Task
– Tag-type that specifies the type of the problem to solve. Can be or .
Constructors
infer_result
()
Creates a new instance of the class with the default property values.
Properties
const
table &
labels
The LaTex Math image. table with the predicted labels.
Default value
: table{}.
Getter & Setter


const table & get_labels() const
auto & set_labels(const table &value)

const
table &
responses
The LaTex Math image. table with the predicted responses.
Default value
: table{}.
Getter & Setter


const table & get_responses() const
auto & set_responses(const table &value)

const
table &
probabilities
A LaTex Math image. table with the predicted class probabilities for each observation.
Getter & Setter


template > const table & get_probabilities() const
template > auto & set_probabilities(const table &value)

Operation
template<typename
Descriptor
> decision_forest::infer_result
infer
(
const
Descriptor &
desc
,
const
decision_forest::infer_input &
input
)
Parameters
  • desc
    – Decision Forest algorithm descriptor
    decision_forest::descriptor
    .
  • input
    – Input data for the inference operation
Preconditions


input.data.is_empty == false

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.