Wireless Network-Ready PCB Defect Detection Reference Implementation

Version: 4.0.0   Published: 03/02/2021  

Last Updated: 03/21/2022

Overview

Deploy an industrial AI application for product quality checks using the Intel® Smart Edge Open platform. Includes support for two types of defect detection: missing components and short circuits due to solder bridge formed during the assembly process. 

To run the reference implementation, you will need to first download and install the Intel® Smart Edge Open Developer Experience Kit.

Once you have installed the Intel® Smart Edge Open Developer Experience Kit, select Configure & Download to download the reference implementation and the software listed below.  

Configure & Download

PCB Defect Detection

Time to Complete

Programming
Language

Available Software

1-1.5 hours

Python*, C++


Intel® Smart Edge Open version 21.12

Intel® Edge Insights for Industrial (EII) version 2.6.2

Target System Requirements

Intel® Smart Edge Open Nodes

  • One of the following processors
    • Intel® Xeon® scalable processor.
    • Intel® Xeon® processor D. 
  • At least 64 GB RAM.
  • At least 265 GB hard drive.
  • An Internet connection.
  • Ubuntu* 20.04 LTS Server.
  • IP camera or pre-recorded video(s)

 

How It Works

The Printed Circuit Board (PCB) defects detection is a reference implementation enabled for detection of defects in the components assembly in a factory environment. The IP Camera will record the video of the assembled boards moving over a conveyor belt. These IP cameras will stream the video to EII video processing applications namely Video Ingestion, Video Analytics, Time Series analysis, etc. using the RTSP streaming protocol. The captured video is analyzed and filters out selected images for further defect analysis. 

The Intel® Smart Edge Open Developer Experience Kit platform infrastructure is used to deploy the application pods of EII performing image processing functionality. The purpose of this reference implementation is to showcase how the Intel® Smart Edge Open Developer Experience Kit platform can be used for deploying Industrial AI applications for product quality check. There are two types of defect detection supported in this demo: missing component detection and short circuit due to solder bridge formed during the assembly process. 

The video file used in this reference implementation is pcb_d2000.avi which is sent from the camera stream pod as RTSP stream. This PCB defects detection reference implementation includes five types of application pods in the Intel® Smart Edge Open Developer Experience Kit Edge Nodes.

Edge Insights for Industrial from Intel is a set of pre-validated ingredients for integrating video and time-series data analytics on edge compute nodes.

architecture diagram
Figure 1: Architecture Diagram

 

  • Etcd 
    The Etcd pod stores the key, value of Video-Ingestion, Video-Analytics, and Visualizer pod access information. It manages the configuration data, state data, and metadata for Kubernetes*.
  • Camera Stream
    The Camera Stream pod simulates IP cameras in the real environment. It is used to send recorded video pcb_d2000.avi file as an RTSP stream like an IP camera. On LTE/5G Network real-time deployment, replace Camera Stream pod with a real IP camera.
  • Video Ingestion
    The Video Ingestion pod is mainly responsible converting the video received from camera stream into frames. It filters the frames and ingests these video frames into the EII stack for further processing and defect analysis.
  • Video Analytics
    The Video Analytics pod receives frames from the Video Ingestion pod, performs classifier converting to Universal Disk Format (UDF), and does the required inferencing on the chosen/support hardware (CPU, VPU) using OpenVINO™ toolkit. In this PCB demo, udfs filter configured as pcb. pcb_filter which will detect a defect on the board that has any solder short or missing part. Video Analytics identifies the defects and marks the defect on the images.
  • Visualizer
    The Visualizer pod mainly displays the frames on GUI display received from Video Analytics pod.

The block diagram below shows the video/image flow through the application.

PCB Image Processing Flow
Figure 2: PCB Image Processing Flow

 

Get Started

These instructions walk you through installing and uninstalling an example PCB Defect Detection reference application using commands. After installation, you will be able to see the application is running on the cluster. The uninstall commands can be used to uninstall the application.

Prerequisites

To run the reference implementation, you will need to first download and install the Intel® Smart Edge Open Developer Experience Kit.

Make sure that the Intel® Smart Edge Open Developer Experience Kit package is installed properly as per installation guide to ensure a smooth installation of PCB Defect Detection RI.

Refer to the link below: Intel® Smart Edge Open Developer Experience Kit Install 

 

Install the Reference Implementation

Select Configure & Download to download the reference implementation and then follow the steps below to install it.  

Configure & Download

  1. Make sure that the Target System Requirements are met properly before proceeding further.  
    • For single-device mode, only one machine is needed. (Both controller and edge node will be on the same device.) 
    • For multi-device mode, make sure you have at least two machines (one for controller and other for Edge Node).
      NOTE: Multi-device mode is not supported in the current release. 
  2. Open a new terminal and login to server as a non-root user. 
    ssh smartedge-open@<server-ip-address>
     

  3. Confirm that the server is configured with ESP Intel® Smart Edge Open Developer Experience Kit as prerequisite. 

  4. Move the downloaded zip package to /home/<non-root-user> folder. 

    mv <path-of-downloaded-directory>/Wireless-network-ready-pcb-defect-detection.zip /home/<non-root-user>

     

  5. Go to the /root directory of the non-root user using the following command and unzip the RI:  
    cd /home/<non-root-user> 
unzip Wireless-network-ready-pcb-defect-detection.zip
  6. Go to Wireless-network-ready-pcb-defect-detection/ directory:  
    cd Wireless-network-ready-pcb-defect-detection
  7. Change permission of the executable edgesoftware file:  
    chmod 755 edgesoftware 
  8. Run the command below to install the Reference Implementation:  
    ./edgesoftware install 

     

  9. During the installation, you will be prompted for the Product Key. The Product Key is contained in the email you received from Intel confirming your download.  
    NOTE: Installation failure logs are available at: /var/log/esb-cli/Wireless-network-ready-pcb-defect-detection/<Component_name>/install.log
     
    Product Key Screen
    Figure 3: Product Key

     
  10. When the installation is complete, you see the message Installation of package complete and the installation status for each module.  
    Install Success Screen
    Figure 4: Installation Successful

 

  1. If Intel® Smart Edge Open Developer Experience Kit is installed, running the following command should show output similar to the image below. All the pods should be either in the running or completed stage. 
      
    kubectl get pods -A
     
    Pods Status Check Screen
    Figure 5: Pods Status

    Note: The status of the Visualizer pod might be CrashLoopBackOff. This is an expected behavior and can be ignored. 

 

  1. If PCB Defect Detection is installed, running the following command should show output as follows:  
    smartedge-open@:/$ kubectl get pods -n smartedge-apps 

NAME                                                  READY   STATUS    RESTARTS   AGE 
deployment-etcd-ui-b594fbdb7-ftfk8            1/1     Running      0          51m 
deployment-video-analytics-86c68fbb5b-mbrgz   1/1     Running      0          51m 
deployment-video-ingestion-5c7ff7c559-cpwfx   1/1     Running      0          51m 
deployment-webvisualizer-676d9cb487-28mss     1/1     Running      0          51m 
ia-etcd-54cdfd9d95-trknn                      2/2     Running      0          51m
    PCB Defect Detection Pods Status
    Figure 6: PCB Defect Detection Pods Status

     

  2. Run the below command to make sure the network policy is created:

    Note: In Intel® Smart Edge Open, the default network policy blocks ingress traffic to all pods, hence the pcb defect detection pod network policy is created to allow the ingress traffic. 
      
    smartedge-open@:/$ kubectl get networkpolicies -n smartedge-apps 

NAME                POD-SELECTOR          AGE 
allow-pcb-ingress   app=webvisualizer     78m
     
    Network Policy Screen
    Figure 7: PCB Defect Detection Network Policy

 

  1. Run the command below to check the Docker* images and their details:  
    smartedge-open@:/$ docker images | grep ia_ 

openedgeinsights/ia_video_ingestion      2.6           7e07ff0b3451   6 weeks ago     2.86GB 
openedgeinsights/ia_web_visualizer       2.6           5bbe38bd3e84   6 weeks ago     2.21GB 
openedgeinsights/ia_video_analytics      2.6           ffe3b44354e1   6 weeks ago     2.82GB 
openedgeinsights/ia_etcd_ui              2.6           cd272efd64dc   6 weeks ago     545MB 
openedgeinsights/ia_etcd_provision       2.6           2683c27dd3c6   2 months ago    197MB 
openedgeinsights/ia_etcd                 2.6           2e8ec01a3d93   2 months ago    119MB

     

  2. List the reference implementation deployed module using the following command:  
    ./edgesoftware list
    PCB Defect Detection Module list
    Figure 8: PCB Defect Detection Module list

     

  3. Uninstall the reference implementation module using the following command: 

    ./edgesoftware uninstall <pcb-module-id>
    Uninstall Success Screen
    Figure 9: Uninstallation Successful 

     

  4. Run the command below to make sure the network policy is deleted after uninstallation: 

    smartedge-open@:/$ kubectl get networkpolicies. 

NAME                     POD-SELECTOR             AGEs 
block-all-ingress        <none>                   3d18h


     

Node Feature Discovery (NFD) Feature

PCB Defect Detection uses the Intel® Distribution of OpenVINO™ toolkit, which is optimized for Intel® processors that support special instructions like AVX512VNNI for optimized performance. The deployment of this application will require the node with this feature supported on the node along with Ubuntu 20.04 OS. This NFD feature ensures to deploy the application on the node supported with these features. 

NFD is installed by Intel® Smart Edge Open Developer Experience Kit and running as two pods on Intel® Smart Edge Open, as shown below.

$ kubectl get pods -A | grep smartedge-system
   
smartedge-system      nfd-release-node-feature-discovery-master-7b94765ccf-9ghjg   1/1    Running   5 (6d18h ago)    7d23h
smartedge-system      nfd-release-node-feature-discovery-worker-dq4x6              1/1    Running   5 (6d18h ago)    7d23h

 

PCB Defect Detection pods scheduled and running successfully on Intel® Smart Edge Open node based on hardware capabilities of Intel® Xeon® Scalable server.

$ kubectl get pods -A | grep smartedge-apps

smartedge-apps        deployment-etcd-ui-6874497b99-khhhz                 1/1     Running   0     3m47s
smartedge-apps        deployment-video-analytics-67f4f95c8c-4z2n6         1/1     Running   0     3m46s
smartedge-apps        deployment-video-ingestion-7b946f6b79-5cbgq         1/1     Running   0     3m47s
smartedge-apps        deployment-webvisualizer-69f98454bf-7q9j2           1/1     Running   0     3m47s
smartedge-apps        ia-etcd-bc6f85bf5-jcpjp                             2/2     Running   0     3m48s

 

The following output shows a description of one PCB Defect Detection pod, which shows that it is running successfully with the NFD feature.

$ kubectl describe pod deployment-video-analytics-67f4f95c8c-4z2n6 -n smartedge-apps
...
...
...
Node-Selectors:              feature.node.kubernetes.io/cpu-cpuid.AVX512VNNI=true
                            feature.node.kubernetes.io/system-os_release.ID=ubuntu
                            feature.node.kubernetes.io/system-os_release.VERSION_ID=20.04
                            feature.node.kubernetes.io/system-os_release.VERSION_ID.major=20
                            feature.node.kubernetes.io/system-os_release.VERSION_ID.minor=04
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                            node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type    Reason     Age   From               Message
----    ------     ----  ----               -------
Normal  Scheduled  23m   default-scheduler  Successfully assigned smartedge-apps/deployment-video-analytics-67f4f95c8c-4z2n6 to esi62
Normal  Pulled     23m   kubelet            Container image "openedgeinsights/ia_video_analytics:2.6.1" already present on machine
Normal  Created    23m   kubelet            Created container ia-video-analytics
Normal  Started    23m   kubelet            Started container ia-video-analytics

 

Public Helm Registry for Helm Charts

Installation of PCB Defect Detection Reference Implementation on Intel® Smart Edge Open is accomplished using Helm charts. Earlier Helm charts used to be a part of Reference Implementation installation package. Now a global Helm repo is issued, so that Reference Implementation Helm charts can be accessible from private and public network. This will speed up and ease the process of introducing updates and their integration with the Reference Implementation.

 

Visualize the Output

To visualize the results, open Google* Chrome and navigate to the links below for respective results: 

Web Visualizer

  • https://<Controller-System-IP-Address>:30009 -DEV Mode
  • https://<Controller-System-IP-Address>:30007 -PROD Mode

Etcd UI

  • https://<Controller-System-IP-Address>:300010

To access the visualizer, log in with the username = admin and password = admin@123

PCB Defect Detection Results

You can view reference input and output videos of the PCB defect detection demo in GitHub. 

Refer to the PCB defects detection results section to cross-verify the results on your setup. 

Following are the reference input and output videos of PCB defect detection demo.

  • Input PCB rtsp stream: The input video file has three PCBs rotating through the screen are sent from the camera stream pod, two of these boards are defective out of three.

  • Output defect detection on PCB: The Visualizer pod showing the output of defect detection on PCB image, i.e. with red square box defect detection marked as PCB board missing component and solder bridge causing a short circuit.

 

 

Screenshot of application results
Figure 10: Application Results

 

Summary and Next Steps

The PCB demo sample application when deployed on the Intel® Smart Edge Open Developer Experience Kit creates an impactful edge computing use case that utilizes the capability of Intel® Smart Edge Open Developer Experience Kit and Edge Insights for Industrial (EII). 

Learn More

To continue learning, see the following guides and software resources:

Troubleshooting

 

Installation Prerequisites

If any issue is faced during installation, ensure that following prerequisites are configured properly.

Hardware Requirements

Make sure you have a fresh ESP Intel® Smart Edge Open Developer Experience Kit installation with the Hardware specified in the Target System Requirements section.

 

Install Python dependent libraries 

Use the following commands: 

pip3 install –-user Cython 
sudo apt-get install python-dev 
sudo apt-get install python3-dev


Confirm the steps below were followed for ESP Intel® Smart Edge Open Developer Experience Kit installation:

  1. Proxy Settings
    If you are behind a proxy network, ensure that proxy addresses are configured in the system:   
    export http_proxy=<proxy-address>:<proxy-port>
export https_proxy=<proxy-address>:<proxy-port> 

     
  2. Ensure the /etc/wgetrc file is configured with required proxy settings as follows:  
    https_proxy=<proxy-address>:<proxy-port> 
http_proxy=<proxy-address>:<proxy-port> 
ftp_proxy =<proxy-address>:<proxy-port> 
use_proxy = on

     

  3. Date and Time 
    Make sure that the date and time are in sync with current local time.  

  4. Create Non-Root User 

    1. To create a user smartedge-open, execute the command:  

      useradd -s /bin/bash -d /home/smartedge-open/ -m -G sudo smartedge-open  
passwd smartedge-open

       

    2. A password for the given user is required: passwd “smartedge-open”  

    3. As some tasks require root privileges, the non-root user needs to have a possibility to become a root: 

      echo “smartedge-open ALL=(ALL) NOPASSWD:ALL” | sudo tee /etc/sudoers.d/smartedge-open

  5. Exchanging SSH keys between hosts
    Exchanging SSH keys between hosts permits a password-less SSH connection from the host running Ansible to the hosts being set up. If password-less SSH is not permitted, follow the instructions below to exchange SSH keys between hosts. Generate and install the public key on your host by logging in as root and executing the below commands: 
             
    In the first command, the Edge Controller host must have a generated SSH key. The SSH key can be generated by executing ssh-keygen and obtaining the key from the output of the command.
    In the second command, the generated key must be copied to every Edge Node host, using the ssh-copy-id command.
     

    ssh-keygen 
ssh-copy-id smartedge-open@<smartedge-open-node-ip-address> 

     

Pods Status Check

Verify that the pods are Ready as well as in Running state using the following command:

kubectl get pods -A

 

Pod Status shows "ImagePullBackOff" for a long time

If the pods are in ImagePullBackOff state for a long time as shown below, run the following commands.

Pod Status ImagePullBackOff

 

Describe the pods using:

kubectl describe pod -n smartedge-apps <pod name>

 

Fetch the image name with the tag and do a manual Docker pull using the commands:

docker login
docker pull <image-name:tag>

 

Pod Status shows "ContainerCreating" for a long time

If pod status shows ContainerCreating or Error or CrashLoopBackOff for 5 minutes or more, run the following commands:

./edgesoftware uninstall -a
./edgesoftware install

 

 

Pod Status shows "pending" for a long time

If only video-analytics pod status is pending for 5 minutes or more, run the following command:

kubectl describe pods -n smartedge-apps

 

In video-analytics pod, if you see the following event due to FailedScheduling, it may be possible that AVX-512VNNI instruction set is not available in system CPU as mentioned in the NFD feature section.

Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 43s (x9 over 7m58s) default-scheduler 0/1 nodes are available: 1 node(s) didn't match Pod's node affinity/selector.

 

If all four pods etcd-ui, video-analytics, video-ingestion and web-visualizer are in pending state for more than 5 minutes, check if the Operating system is Ubuntu 20.04 LTS or not. 

 

Uninstallation Failure   

If uninstall command (./edgesoftware uninstall < module id >) fails, manually uninstall PCB Defect Detection deployment and network policy, which are specific to pcb defect detection pod: 

helm uninstall pods -n smartedge-apps   
kubectl delete networkpolicy allow-pcb-ingress

Installation and Debug Log Info File Path 

Installation log info of Intel® Smart Edge Open Developer Experience Kit will be available at:  

/var/log/esb-cli/Smart_Edge_Open_Developer_Experience_Kits_2.0.0/Smart_Edge_Open_Developer_Experience_Kits/install.log 

Installation log info of Wireless Network Ready PCB Defect Detection module will be available at: 

/var/log/esb-cli/Wireless_NetworkReady_PCB_defect_detection_4.0.0/WirelessNetwork_Ready_PCB_defect_detection/install.log

Execute the command below to consolidate a list of the log files in tar.gz compressed format, e.g., Wireless_NetworkReady_PCB_defect_detection.tar.gz.  

tar -czvf Wireless_NetworkReady_PCB_defect_detection.tar.gz /var/log/esb-cli/Wireless_NetworkReady_PCB_defect_detection_4.0.0/Wireless_NetworkReady_PCB_defect_detection/ /var/log/esb-cli/Smart_Edge_Open_Developer_Experience_Kits_2.0.0/Smart_Edge_Open_Developer_Experience_Kits/

 

Support Forum 

If you're unable to resolve your issues, contact the Support Forum.  

  • Make sure you have an active internet connection during the full installation. If you lose Internet connectivity at any time, the installation might fail.

  • Make sure you are using a fresh installation. Earlier software, especially Docker* and Docker Compose*, can cause issues.

  • Use the kubectl commands like describe, log, get etc. to analyze the problems.

  • Ensure the proxy settings are properly set.

  • In case of Docker pull rate limit issue, we recommend you use the Premium/PRO Docker teams to get unlimited docker pull support.

 

 

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.