How it Works

The application uses the inference engine included in the Intel® Distribution of OpenVINO™ toolkit. The reference implementation shows an example of how segmentation networks can be leveraged in industrial quality inspection applications.

• Surface inspection video: Sample video that is used to simulate a surface defect inspection camera stream. The video includes different types of surface defects.

• Inference Module: Ingests the frames from the camera stream and performs inference on each frame to segment defects. The rate at which each frame is detected is also recorded and displayed on the output Visualizer window of the Edge Insights for Industrial software stack.

All communication between modules occurs over the Edge Insights for Industrial Message Bus. This reference implementation is a sample application that can be executed using the established pipeline.

Get Started

The Industrial Surface Defect Detection reference implementation is a plug-and-play application developed for the Edge Insights for Industrial package.

Industrial Surface Defect Detection is a reference implementation that can be executed as part of the Edge Insights for Industrial package. If you have not installed that package yet, you can download it here and then follow the Edge Insights for Industrialinstallation instructions.

Step 1: Add Surface Defect Detection UDF Components

1. Copy the Industrial Surface Defect Detection UDF file to the Edge Insights for Industrial UDFs directory:

   # Set Edge Insights for Industrial home path as environment variable
   EII_HOME=<path_to_install_dir/IEdgeInsights/>
   
   
   # Copy Surface Defect Detection UDF file to the UDFs directory
   cp -r PySurfaceDefectDetection $EII_HOME/CustomUdfs/.

2. Modify the Visualizer and WebVisualizer configuration files to add Industrial Surface Defect Detection to the IEdgeInsights pipeline:

   NOTE:

   We recommend that you make copies of the original $EII_HOME/Visualizer/config.json and $EII_HOME/WebVisualizer/config.json files before proceeding. These files needed to be rewritten to modify the pipeline to use the Industrial Surface Defect Detection application files instead of the default PCB demo application.

   1. Make copies of the original Visualizer and WebVisualizer config.json files:

      mv $EII_HOME/Visualizer/config.json $EII_HOME/Visualizer/config.json_original
      mv $EII_HOME/WebVisualizer/config.json $EII_HOME/WebVisualizer/config.json_original

   2. Copy the new Visualizer and WebVisualizer config.json files to execute the Industrial Surface Defect Detection application.

      Optionally, edit the PySurfaceDefectDetection/config.json to use the inference hardware of choice [CPU/GPU/Myriad/HDDL]. Default is CPU.

      NOTE:

      The deep learning model used in the reference implementation is a deep model, and hence would take a few minutes for the initial loading into an accelerator (Myriad/HDDL) memory.

      cp vis_config.json $EII_HOME/Visualizer/config.json
      cp webvis_config.json $EII_HOME/WebVisualizer/config.json

   3. Copy the custom UDF yml file to include the custom surface defect detection UDF container:

      cp video-streaming-surfacedefectUDF.yml $EII_HOME/build/usecases/.

      Download the DAGM data and convert the Class1-Class6 test images to video defect_segmentation.avi with the convert code:

      import cv2
      import glob
      
      
      img_path = 'Your/DAGM/TEST/IMAGE/PATH'
      img_list = glob.glob(img_path + '*.PNG')
      fourcc = cv2.VideoWriter_fourcc(*'XVID')
      writer = cv2.VideoWriter('surface_defect.avi', fourcc, 2,
                                   (512, 512))
      for imgi in img_list:
          img = cv2.imread(imgi)
          writer.write(img)
      writer.release()

3. Follow the Edge Insights for Industrial instructions to provision and launch the containers:

   # Configure the IEdgeInsights pipeline to use custom UDF, Visualizer & Web Visualizer containers
   cd $EII_HOME/build/
   sudo python3 builder.py -f usecases/video-streaming-surfacedefectUDF.yml
   
   
   # Provision
   cd $EII_HOME/build/provision/
   sudo -E ./provision.sh ../docker-compose.yml
   
   
   # Build and run
   cd $EII_HOME/build/
   docker-compose -f docker-compose-build.yml build python_surface_defect_detection
   docker-compose up -d
   
   
   # Enable visualizer display
   xhost +

   The output will look similiar to:

   

Step 2: Check the Reference Implementation

The Edge Insights for Industrial builds the custom UDF, Visualizer and WebVisualizer containers. The custom UDF container streams the sample video and performs segmentation on each frame to detect defects. Once the frame has been segmented, the image is displayed on the Visualizer output window with the segmentation result and inference time.

Execute the following command to ensure all the containers are running without errors:

sudo docker ps

Check for Success

If it was successful, the results will be similar to:

WebVisualizer Output

The sample video has different types of defects that can be monitored using the Web Visualizer:

(Optional) Stop the Vision Pipline

To stop the Edge Insights for Industrial software, execute the following command:

cd $EII_HOME/build/
docker-compose down

To revert to the default PCB Demo application, move the Visualizer and WebVisualizer config.json files to the original copies [Step 1.2.a] and rebuild using the instructions in Step 1.3.

(Optional) Standalone Testing

Testing the Industrial Surface Defect Detection reference implementation outside of the Edge Insights for Industrial deployment environment requires OpenVINO™ to be installed on the development system. Please follow instructions on the OpenVINO™ website for installation steps for different inference hardware (CPU/Myriad/HDDL). The reference model is training with DAGM dataset Class1~Class6, you can download it here.

To execute the application on the host development system, follow the instructions below:

# Install dependencies
pip install opencv-python==4.4.0.42


# Set OpenVINO env variables
source /opt/intel/openvino/bin/setupvars.sh -pyver 3.6


# Run inference on single test image
python3 defect_segmentation.py \
        --model <path/to/model/XML/file> \
        --label <path/to/labels/txt/file> \
        --image <path/to/input/test/image>


        Ex.:
        python3 surface_defect/defect_segmentation.py \
            --model surface_defect/ref/model.xml\
            --image sample_data/Class1_0837.png


        Expected output:
            Image : ../sample_data/Class1_0837.png
            INFO:DEFECT_SEGMENTATION:Prediction : 1
            INFO:DEFECT_SEGMENTATION:Inference time : 0.0775909423828125 sec
            ***********************************************************


# Run inference on director of test images
python3 defect_segmentation.py \
        --model <path/to/model/XML/file> \
        --label <path/to/labels/txt/file> \
        --dir <path/to/dir/of/test/images>


        Expected output:
            Image : ../sample_data/Class6_None-Defect.png
            INFO:DEFECT_SEGMENTATION:Prediction : 0
            INFO:DEFECT_SEGMENTATION:Inference time : 0.07543802261352539 sec
            ***********************************************************


            Image : ../sample_data/Class4_0836.png
            INFO:DEFECT_SEGMENTATION:Prediction : 1
            INFO:DEFECT_SEGMENTATION:Inference time : 0.03906846046447754 sec
            ***********************************************************
            ...

Summary and Next Steps

In this tutorial, you successfully ran the Industrial Surface Defect Detection application and displayed the result using the Edge Insights for Industrial Visualizer output.

To get access to the deep learning training algorithm that was used to generate the Surface Defect Detection model, please contact your Intel account manager.

As a next step, see the Weld Porosity Detection tutorial.