Intel® Software Network

Using the Test Prioritization Tool is a simple 3-step process.

The table below shows results from a hypothetical, application called myApp.c. One of three tests can be eliminated with no reduction in testing coverage.

Output from the Test-Prioritization Tool

Num	% Rate Coverage	% Block Coverage	% Function Coverage	Test Name @ Options
1	87.50	45.65	37.50	Test3.dpi
2	100.00	52.17	50.00	Test2.dpi

Total number of tests = 3 Total block coverage ~ 52.17 Total function coverage ~ 50.00

In this case, the Test-Prioritization Tool provided the following information:

• By running all three tests, developers achieve 52.17% block coverage and 50.00% function coverage.
• Test3 alone covers 45.65% of the basic blocks of the application, which is 87.50% of the total block coverage achieved from all three tests.
• By adding Test2, developers achieve cumulative block coverage of 52.17%, or 100% of the total block coverage of Test1, Test2, and Test3.
• Elimination of Test1 has no negative impact on the total block coverage.

In addition to testing for the minimum number of tests, developers can use the tool to complete test coverage in the minimum amount of time. Assume three different tests ran as shown below:

Test1.dpi 00:00:60:35 Test2.dpi 00:00:10:15 Test3.dpi 00:00:30:45

When running the Test-Prioritization Tool to minimize the execution time, we get the following results:

Sample Output from the Intel Compilers Test-Prioritization Tool

Num	Elapsed Time (min.)	% Rate Coverage	% Block Coverage	% Function Coverage	Test Name @ Options
1	10:15	75.00	39.13	25.00	Test2.dpi
2	41:00	100.00	52.17	50.00	Test3.dpi

Total number of tests = 3 Total block coverage ~ 52.17 Total function coverage ~ 50.00 Total execution time = 1:41:35

The results indicate that running all tests sequentially requires one hour, 41 minutes, and 35 seconds, while the selected tests achieve the same total block coverage in only 41 minutes. When prioritization is based in minimizing time (by running first Test2, then Test3), note that the order of tests may differ when prioritization is based on minimizing the number of tests (by running first Test3, then Test2). In the example shown in the table above, Test2 provides the highest coverage per execution time. Therefore, it is selected as the first test to run.

The Test Prioritization Tool can be directed to "cut off" (exit) when it reaches a certain level of basic block coverage. If the tool is told to run until it reaches 85% coverage, only Test3 is selected in this example since it achieves 45.65% block coverage. This is 87.50% of the total block coverage that can be achieved from all three tests.