Processing an Image from Edge to Edge using Intel® Integrated Performance Primitives

Introduction

Intel® Integrated Performance Primitives (Intel® IPP) Image Processing domain is extensive, ready-to-use, image functions that are highly optimized for diverse Intel® architectures.The functions are highly optimized using Intel® Streaming SIMD Extensions 2, Intel® Advanced Vector Extensions 2 (Intel® AVX2), and Intel® Advanced Vector Extensions 512 (Intel® AVX-512) instruction sets.

Take visual information and convert it into usable data for further analysis and decision-making. As the volume of imaging information captured by vision systems rises, image processing converts image arrays into manageable units. In this article, we will be looking into Intel® IPP Image Filtering functions and processing an image from edge to edge with tips on resolving memory violations that occur when using Intel® IPP Filtering functions.

Intel® IPP Library is available as part of the Intel® oneAPI Base Toolkit.

Intel® IPP Image Filtering Functions

Intel® IPP Image filtering functions assume that for each pixel being processed, adjacent pixels also exist. If required adjacent pixel values are missing, a memory violation or access violation error will occur. Intel® IPP functions provides special argument IppiBorderType borderType for edge pixels.

The borderType can be one of types {ippBorderInMem, ippBorderRepl,ippBorderMirror,ippBorderConst,..}. See full border list either in Intel® IPP documentation or in “ipptypes.h” in the Intel® IPP package.

There are two ways to work with borders:

• Allocate image with additional pixels and fill manually or using ippiCopyReplicateBorder/ ippiCopyConstBorder/ ippiCopyWrapBorder. Pass borderType= ippBorderInMem. It is the most preferable way for functions by performance reasons.
• Allocate equal size of the source and destination images and use borderType= ippBorderRepl/Const/Mirror. Functions will create “virtual” borders pixels. But this approach has some overhead. Particularly, for small images.
   

Process the edges with additional pixels. Process the edges with an adjusted starting point, and ROI

Depending on image size and filter size, adjusting the anchor point and ROI may eliminate the need for adding data to the image border.

Code Example: In this sample. we ignore the border pixels, our ROI only include the area one pixel from the border.

void printImg_8u(char* s, Ipp8u* pSrc, int srcStep, IppiSize roiSize)
{
    int i, j;
    printf("%s\n", s);
    for (j = 0; j < roiSize.height; j++) {
        for (i = 0; i < roiSize.width; i++) {
            printf("%3d ", pSrc[i]);
        }
        pSrc = (Ipp8u*)((Ipp8u*)pSrc + srcStep);
        printf("\n");
    }
    printf("\n");
}
void filtermax()
{
    IppiSize imgSize = { 6, 6 };/* by 1 pixel from each side*/
    IppiSize roiSize   = { 4, 4 };

    int srcStep = imgSize.width  * sizeof(Ipp8u);
    int size    = imgSize.height * srcStep;

    Ipp8u* src = (Ipp8u*)malloc(size);
    ippiImageJaehne_8u_C1R(src, srcStep, imgSize);
    int dstStep = roiSize.width * sizeof(Ipp8u);
    int dstSize = roiSize.width*dstStep;
    Ipp8u* dst = (Ipp8u*)malloc(dstSize);

    IppStatus status;
    IppiSize maskSize = { 3,3 };
    int specSize, bufferSize, initSize;
    Ipp8u* pBuffer;
    
    status = ippiFilterMaxBorderGetBufferSize(roiSize, maskSize, ipp8u, 1, &bufferSize);
    pBuffer = (Ipp8u*)malloc(bufferSize);
   /*adjust pointer shifting it 1 pixel down and right*/
    status = ippiFilterMaxBorder_8u_C1R(src + srcStep / sizeof(Ipp8u) + 1, srcStep,
        dst, dstStep, roiSize, maskSize, ippBorderInMem, 0, pBuffer);
    printImg_8u("src", src, srcStep, imgSize);
    printImg_8u("dst", dst, dstStep, roiSize);
    free(pBuffer);
    free(dst);
    free(src);
}

src
111 230 255 255 230 111
230 246 206 206 246 230
255 206 145 145 206 255
255 206 145 145 206 255
230 246 206 206 246 230
111 230 255 255 230 111

dst
255 255 255 255
255 246 246 255
255 246 246 255
255 255 255 255

If the filtering operation occurs on a section of the source image (ROI), then the necessity of adding border pixels depends upon the ROI size and position. If the image edges are part of the ROI, border pixels must be added (see below).

Duplicate the edge pixels around the image

In this case, you need to:

1. Determine which additional border pixels are required
2. Create/define these pixels

Special functions have been developed for extending image edges:

Code Example: In this sample, we use ippiCopyReplicateBorder_8u_C1IR() function to extend the image border. This ensures that all of the pixels in the ROI have neighbor pixels to process.

void filtermax_copyconst()
{
    IppiSize imgSize = { 6, 6 };
    IppiSize roiSize = { 4, 4 };
    int srcStep = imgSize.width * sizeof(Ipp8u);
    int size = imgSize.height * srcStep;
    Ipp8u* src = (Ipp8u*)malloc(size);
    int dstStep = roiSize.width * sizeof(Ipp8u);
    int dstSize = roiSize.width*dstStep;
    Ipp8u* dst = (Ipp8u*)malloc(dstSize);
    int topBorderHeight = 1;
    int leftBorderWidth = 1;
    ippiImageJaehne_8u_C1R(dst, dstStep, roiSize);
    ippiCopyConstBorder_8u_C1R(dst, dstStep, roiSize, src, srcStep, imgSize, topBorderHeight, leftBorderWidth, 255);
    IppStatus status;
    IppiSize maskSize = { 3,3 };
    int specSize, bufferSize, initSize;
    Ipp8u* pBuffer;
    status = ippiFilterMaxBorderGetBufferSize(roiSize, maskSize, ipp8u, 1, &bufferSize);
    pBuffer = (Ipp8u*)malloc(bufferSize);
    status = ippiFilterMaxBorder_8u_C1R(src + srcStep / sizeof(Ipp8u) + 1, srcStep,
        dst, dstStep, roiSize, maskSize, ippBorderInMem, 0, pBuffer);
    printImg_8u("src", src, srcStep, imgSize);
    printImg_8u("dst", dst, dstStep, roiSize);
    free(pBuffer);
    free(dst);
    free(src);
}
src
255 255 255 255 255 255
255 254 234 234 254 255
255 234 153 153 234 255
255 234 153 153 234 255
255 254 234 234 254 255
255 255 255 255 255 255

dst
255 255 255 255
255 254 254 255
255 254 254 255
255 255 255 255 

Use predefined border type as function argument and the same size of input and output images

To avoid creating image with additional pixel it is possible to use borderType= ippBorderConst.
borderValue=255 is used together with ippBorderConst. 

void filtermax_borderconst()
{
    IppiSize roiSize = { 4, 4 };
    IppiSize imgSize = roiSize;

    int srcStep = imgSize.width * sizeof(Ipp8u);
    int size = imgSize.height * srcStep;

    Ipp8u* src = (Ipp8u*)malloc(size);
    int dstStep = roiSize.width * sizeof(Ipp8u);
    int dstSize = roiSize.width*dstStep;
    Ipp8u* dst = (Ipp8u*)malloc(dstSize);

    ippiImageJaehne_8u_C1R(src, srcStep, roiSize);


    IppStatus status;
    IppiSize maskSize = { 3,3 };
    int specSize, bufferSize, initSize;
    Ipp8u* pBuffer;

    status = ippiFilterMaxBorderGetBufferSize(roiSize, maskSize, ipp8u, 1, &bufferSize);
    pBuffer = (Ipp8u*)malloc(bufferSize);
    status = ippiFilterMaxBorder_8u_C1R(src, srcStep,
        dst, dstStep, roiSize, maskSize, ippBorderConst, 255, pBuffer);


    printImg_8u("src", src, srcStep, imgSize);
    printImg_8u("dst", dst, dstStep, roiSize);
    free(pBuffer);
    free(dst);
    free(src);
}
src
254 234 234 254
234 153 153 234
234 153 153 234
254 234 234 254

dst
255 255 255 255
255 254 254 255
255 254 254 255
255 255 255 255

Output is identical with previous example of ippiCopyConstBoder.

Related Topic

Some of Intel® IPP Image processing function also require external working buffer (ippiResizeCubic_8u) or special state structure as an input parameter in order to easily memory management. For example, the Intel® IPP ippiDilateBorder requires one IppiMorphState structure as parameter. You can use the following sequence to call the ippiDilateBorder function:

1. Call function ippiMorphologyBorderGetSize to get the size requirement for morphology state structure and work buffer pBuffer of processing function ippiDilateBorder.
2. Ensure that the required memory space for morphology state structure is properly allocated.
3. Call the ippiMorphologyBorderInit function to initiate morphology state structure.
4. It is better to place buffers size calculations, memory allocations and initialization functions to init part of your applications because many Intel® IPP initialization function calculates special tables, in some cases using transcendental math functions.
5. Use the ippiDilateBorder function in your application.
6. Free the memory allocated for morphology state structure

Here is an example of ippiDilateBorder function use:

void morph()
{
    IppStatus status;
    Ipp8u x[7 * 5], y[11 * 9];
    IppiSize roi = { 7,5 }, masksize = { 3,3 };
    Ipp8u Mask[3 * 3] = { 1,1,1,1,1,1,1,1,1 };
    ippiImageJaehne_8u_C1R(x, 7, roi);

    IppiBorderType borderType = ippBorderInMem;
    Ipp8u borderValue = 0;
    int specSize, bufferSize;
    IppiMorphState* pSpec;
    Ipp8u* pBuffer;

    status = ippiMorphologyBorderGetSize_8u_C1R(roi, masksize, &specSize, &bufferSize);
    pBuffer = malloc(bufferSize);
    pSpec = (IppiMorphState*)malloc(specSize);
    status = ippiMorphologyBorderInit_8u_C1R(roi, Mask, masksize, pSpec, pBuffer);
    status = ippiDilateBorder_8u_C1R(x, 7, y, 11, roi, borderType, borderValue, pSpec, pBuffer);

    printImg_8u("x", x, 7,roi);
    printImg_8u("y", y, 11, roi);

    free(pSpec);
    free(pBuffer);
    
}

Note the parameters in functions : srcStep/dstStep. It is the distance in bytes of image row. It depends on your array memory layout and data type.  In most of case, it is equal to the image width*sizeof(DataType)*numChannels.  But sometimes, it is not, especially for bmp images which required 4 bytes aligned. The ippiMalloc returns 64 bytes aligned steps correspondingly. The 64b aligned origin of images and distance between lines are preferable by performance reasons too. So please take care when use srcStep or shift the pointer by srcStep.

Notices & Disclaimers

Intel technologies may require enabled hardware, software or service activation.

No product or component can be absolutely secure. 

Your costs and results may vary. 

© Intel Corporation.  Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries.  Other names and brands may be claimed as the property of others.