Wireless Network-Ready PCB Defect Detection Reference Implementation

Version: 5.0.0   Published: 03/02/2021  

Last Updated: 08/29/2022

Overview

Deploy an industrial AI application for product quality checks using the Intel® Smart Edge Open platform. Includes support for two types of defect detection: missing components and short circuits due to solder bridge formed during the assembly process. 

To run the reference implementation, you will need to first download and install the Intel® Smart Edge Open Developer Experience Kit.

Once you have installed the Intel® Smart Edge Open Developer Experience Kit, select Configure & Download to download the reference implementation and the software listed below.  

Configure & Download

PCB Defect Detection

 

  • Time to Complete: 1-1.5 hours
  • Programming Language: Python*, C++
  • Available Software: 
    • Intel® Smart Edge Open version 22.03 or above
    • Intel® Open Edge Insights version 3.0.0

 

Target System Requirements

Intel® Smart Edge Open Nodes

  • One of the following processors
    • Intel® Xeon® Scalable processor.
    • Intel® Xeon® D processor.
  • At least 64 GB RAM.
  • At least 265 GB hard drive.
  • An Internet connection.
  • Ubuntu* 20.04 LTS Server.
  • IP camera or pre-recorded video(s)

 

How It Works

The Printed Circuit Board (PCB) defects detection is a reference implementation enabled for detection of defects in the components assembly in a factory environment. The IP Camera will record the video of the assembled boards moving over a conveyor belt. These IP cameras will stream the video to Open Edge Insights (OEI) video processing applications namely Video Ingestion, Video Analytics, Time Series analysis, etc. using the RTSP streaming protocol. The captured video is analyzed and filters out selected images for further defect analysis. 

The Intel® Smart Edge Open Developer Experience Kit platform infrastructure is used to deploy the application pods of OEI performing image processing functionality. The purpose of this reference implementation is to showcase how the Intel® Smart Edge Open Developer Experience Kit platform can be used for deploying Industrial AI applications for product quality check. There are two types of defect detection supported in this demo: missing component detection and short circuit due to solder bridge formed during the assembly process. 

The video file used in this reference implementation is pcb_d2000.avi which is sent from the camera stream pod as RTSP stream. This PCB defects detection reference implementation includes five types of application pods in the Intel® Smart Edge Open Developer Experience Kit Edge Nodes.

Edge Insights for Industrial from Intel is a set of pre-validated ingredients for integrating video and time-series data analytics on edge compute nodes.

The architecture is represented by a complex block diagram.
Figure 1: Architecture Diagram

 

  • Config Manager Agent 
    Config Manager Agent is an OEI service responsible to put the OEI services configs to the OEI config manager data store. It creates required provisioning folders with the right permissions needed for other OEI services via volume mounts.
  • Camera Stream
    The Camera Stream pod simulates IP cameras in the real environment. It is used to send recorded video pcb_d2000.avi file as an RTSP stream like an IP camera. On LTE/5G Network real-time deployment, replace Camera Stream pod with a real IP camera.
  • Video Ingestion
    The Video Ingestion pod is mainly responsible converting the video received from camera stream into frames. It filters the frames and ingests these video frames into the OEI stack for further processing and defect analysis.
  • Video Analytics
    The Video Analytics pod receives frames from the Video Ingestion pod, performs classifier converting to Universal Disk Format (UDF), and does the required inferencing on the chosen/support hardware (CPU, VPU) using OpenVINO™ toolkit. In this PCB demo, udfs filter configured as pcb. pcb_filter which will detect a defect on the board that has any solder short or missing part. Video Analytics identifies the defects and marks the defect on the images.
  • Web Visualizer
    The Web Visualizer pod mainly displays the frames on GUI display received from Video Analytics pod.

The block diagram below shows the video/image flow through the application.

A block diagram showing the PCB Image processing flow from left to right.
Figure 2: PCB Image Processing Flow

 

Get Started

These instructions walk you through installing and uninstalling an example PCB Defect Detection reference application using commands. After installation, you will be able to see the application is running on the cluster. The uninstall commands can be used to uninstall the application.

Installation Prerequisites 

To run the reference implementation, you will need to first download and install the Intel® Smart Edge Open Developer Experience Kit.

Ensure that the following conditions are met properly to ensure a smooth installation process for a reference implementation done through Edge Software Provisioner (ESP) Intel® Smart Edge Open Developer Experience Kit package.

 

Hardware Requirements

Make sure you have the hardware as specified in the Target System Requirements section for fresh ESP Intel® Smart Edge Open Developer Experience Kit installation.


Ensure that the steps below are completed in the ESP Intel® Smart Edge Open Developer Experience Kit installation:

  1. Proxy Settings
    If you are behind a proxy network, ensure that proxy addresses are configured in the system:   
    export http_proxy=<proxy-address>:<proxy-port>
export https_proxy=<proxy-address>:<proxy-port> 

     
  2. Ensure the /etc/wgetrc file is configured with required proxy settings as follows:  
    https_proxy=<proxy-address>:<proxy-port> 
http_proxy=<proxy-address>:<proxy-port> 
ftp_proxy =<proxy-address>:<proxy-port> 
use_proxy = on

     

  3. Date and Time 
    Make sure that the date and time are in sync with current local time.  

  4. Create Non-Root User 
    Make sure that non-root user with name and password smartedge-open and with sudoers permission is created. If not available, follow the instructions below.

    1. To create a user smartedge-open, execute the command:  

      useradd -s /bin/bash -d /home/smartedge-open/ -m -G sudo smartedge-open
passwd smartedge-open

       

    2. A password for the given user is required:

      passwd "smartedge-open"

       

    3. As some tasks require root privileges, the non-root user needs to have a possibility to become a root: 

      echo "smartedge-open ALL=(ALL) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/smartedge-open

       

  5. Exchanging SSH keys between hosts
    Exchanging SSH keys between hosts permits a password-less SSH connection from the host running Ansible to the hosts being set up. If password-less SSH is not permitted, follow the instructions below to exchange SSH keys between hosts. Generate and install the public key on your host by logging in as root and executing the below commands: 
             
    In the first command, the Edge Controller host must have a generated SSH key. The SSH key can be generated by executing ssh-keygen and obtaining the key from the output of the command.
    In the second command, the generated key must be copied to every Edge Node host, using the ssh-copy-id command.
     

    ssh-keygen 
ssh-copy-id smartedge-open@<smartedge-open-node-ip-address> 

 

Install Python dependent libraries 

Use the following commands: 

pip3 install –-user Cython 
sudo apt-get install python-dev 
sudo apt-get install python3-dev

 

Install the Reference Implementation

Select Configure & Download to download the reference implementation and then follow the steps below to install it.  

Configure & Download

  1. Make sure that the Target System Requirements are met properly before proceeding further.  
    • For single-device mode, only one machine is needed. (Both controller and edge node will be on the same device.) 
    • For multi-device mode, make sure you have at least two machines (one for controller and other for Edge Node).
      NOTE: Multi-device mode is not supported in the current release. 
  2. Open a new terminal and login to server as a non-root user. 
    ssh smartedge-open@<server-ip-address>
     

  3. Confirm that the server is configured with ESP Intel® Smart Edge Open Developer Experience Kit as prerequisite. 

  4. Move the downloaded zip package to /home/<non-root-user> folder. 

    mv <path-of-downloaded-directory>/Wireless-network-ready-pcb-defect-detection.zip /home/<non-root-user>

     

  5. Go to the /root directory of the non-root user using the following command and unzip the RI:  
    cd /home/<non-root-user> 
unzip Wireless-network-ready-pcb-defect-detection.zip
  6. Go to Wireless-network-ready-pcb-defect-detection/ directory:  
    cd Wireless-network-ready-pcb-defect-detection
  7. Change permission of the executable edgesoftware file:  
    chmod 755 edgesoftware 
  8. Run the command below to install the Reference Implementation:  
    ./edgesoftware install 

     

  9. Installation of the package starts.
     
    A console window showing system output during the install process.
    Figure 3: Installation Start Screen

     
  10. When the installation is complete, you see the message Installation of package complete and the installation status for each module.  

    NOTE: Installation failure logs will be available at: /var/log/esb-cli/Wireless_NetworkReady_PCB_defect_detection_<version>/Wireless_NetworkReady_PCB_defect_detection/install.log
    Where <version> is the package version downloaded.
     
    A console window showing system output during the install process. At the end of the process, the system displays the message “Installation of package complete” and the installation status for each module.
    Figure 4: Installation Successful

 

  1. If Intel® Smart Edge Open Developer Experience Kit is installed, running the following command should show output similar to the image below. All the pods should be either in the running or completed stage. 
      
    kubectl get pods -A
     
    A console window showing system output after running the “kubectl get pods” command. The system displays a list of all the pods and the pod status. The expected status is “Running”.
    Figure 5: Pods Status

    Note: The status of the Visualizer pod might be CrashLoopBackOff. This is an expected behavior and can be ignored. 

 

  1. If PCB Defect Detection is installed, running the following command should show output as follows:  
    $ smartedge-open@host:/$ kubectl get pods -n smartedge-apps

NAME                                        		  READY    STATUS    RESTARTS      AGE
deployment-etcd-ui-b6f9868f5-ll9nc            	       1/1     Running      0          35m
deployment-video-analytics-5749dbcbc9-554rf            1/1     Running      0          35m
deployment-video-ingestion-7485bdff74-2jbbt            1/1     Running      0          35m
deployment-webvisualizer-76496f95d6-rzpfp              1/1     Running      0          35m
ia-configmgr-agent-6f7c488fd5-hk5sw            	       1/1     Running      0          35m
    A console window showing system output after running the “kubectl get pods” command. The system displays a list of all the pods and the pod status. The expected status is “Running”.
    Figure 6: PCB Defect Detection Pods Status

     

  2. Run the below command to make sure the network policy is created:

    Note: In Intel® Smart Edge Open, the default network policy blocks ingress traffic to all pods, hence the pcb defect detection pod network policy is created to allow the ingress traffic. 
      
    $ smartedge-open@:/$ kubectl get networkpolicies -n smartedge-apps

NAME                           POD-SELECTOR        AGE
deny-from-other-namespaces     <none>              26d
etcd-ui-network-policy         app=etcd-ui         17m
webvisualizer-network-policy   app=webvisualizer   17m
     
    A console window showing system output with a list of Network Policies.
    Figure 7: PCB Defect Detection Network Policy

 

  1. Run the command below to check the Docker* images and their details:  
    $ smartedge-open@host:/$ docker images | grep ia_

openedgeinsights/ia_web_visualizer                       3.0           dc770e4abb2d   5 weeks ago     2.77GB
openedgeinsights/ia_video_analytics                      3.0           ae2d02273229   5 weeks ago     3.05GB
openedgeinsights/ia_video_ingestion                      3.0           ac31dafabc77   5 weeks ago     3.3GB
openedgeinsights/ia_etcd_ui                              3.0           939cb1e5db69   5 weeks ago     1.11GB
openedgeinsights/ia_configmgr_agent                      3.0           278d619c406c   6 weeks ago     268MB

     

  2. List the reference implementation deployed module using the following command:  
    ./edgesoftware list
    A console window showing the output of the "edgesoftware list" command. The installed modules are listed.
    Figure 8: PCB Defect Detection Module List

     

  3. Uninstall the reference implementation module using the following command: 

    ./edgesoftware uninstall <pcb-module-id>
    A console window showing the output of the "edgesoftware list" and "edgesoftware uninstall" commands. First, the system lists the installed modules. Next, the system displays output during the uninstall process. At the end of the process, the system displays the message “Uninstall finished” and the uninstallation status for each module.
    Figure 9: Uninstallation Successful 

     

  4. Run the command below to make sure the network policy is deleted after uninstallation: 

    smartedge-open@:/$ kubectl get networkpolicies. 

NAME                     POD-SELECTOR             AGEs 
block-all-ingress        <none>                   32d


     

Node Feature Discovery (NFD)

PCB Defect Detection uses the Intel® Distribution of OpenVINO™ toolkit, which is optimized for Intel® processors that support special instructions like AVX512VNNI for optimized performance. The deployment of this application will require the node with this feature supported on the node along with Ubuntu 20.04 OS. This NFD feature ensures to deploy the application on the node supported with these features. 

NFD is installed by Intel® Smart Edge Open Developer Experience Kit and running as two pods on Intel® Smart Edge Open, as shown below.

$ kubectl get pods -A | grep smartedge-system
   
smartedge-system       nfd-release-node-feature-discovery-master-7b94765ccf-9ghjg   1/1     Running   5 (83d ago)    85d
smartedge-system       nfd-release-node-feature-discovery-worker-dq4x6              1/1     Running   5 (83d ago)    85d

 

PCB Defect Detection pods scheduled and running successfully on Intel® Smart Edge Open node based on hardware capabilities of Intel® Xeon® Scalable server.

$ kubectl get pods -A | grep smartedge-apps

smartedge-apps       deployment-etcd-ui-b6f9868f5-ll9nc                   1/1     Running   0              54m
smartedge-apps       deployment-video-analytics-5749dbcbc9-554rf          1/1     Running   0              54m
smartedge-apps       deployment-video-ingestion-7485bdff74-2jbbt          1/1     Running   0              54m
smartedge-apps       deployment-webvisualizer-76496f95d6-rzpfp            1/1     Running   0              54m
smartedge-apps       ia-configmgr-agent-6f7c488fd5-hk5sw                  1/1     Running   0              54m

 

The following output shows a description of one PCB Defect Detection pod, which shows that it is running successfully with the NFD feature.

$ kubectl describe pod deployment-video-analytics-5749dbcbc9-554rf -n smartedge-apps
...
...
...

Node-Selectors:           feature.node.kubernetes.io/cpu-cpuid.AVX512VNNI=true
                          feature.node.kubernetes.io/system-os_release.ID=ubuntu
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  56m   default-scheduler  Successfully assigned smartedge-apps/deployment-video-analytics-5749dbcbc9-554rf to esi62
  Normal  Pulled     56m   kubelet            Container image "openedgeinsights/ia_video_analytics:3.0" already present on machine
  Normal  Created    56m   kubelet            Created container ia-video-analytics
  Normal  Started    56m   kubelet            Started container ia-video-analytics

 

 

 

Multi-Tenancy and Multi-Verticality Feature


Besides covering multiple IoT verticals, the RIs with Intel® Smart Edge Open increase time-to-market for Independent Software Vendors (ISVs) and provide a reference for edge solution builders looking to deploy multi-tenant and multi-vertical services on the edge platform. Intel® Smart Edge Open via the Intel® Smart Edge Developer Experience Kit offers support for multiple software tools/frameworks (including OEI, OpenVINO™, WebRTC) and diverse, multi-segment Reference Implementations to run simultaneously on the same edge platform) showcasing the foundational principals of interoperability across Intel offerings and assets.

Multiple Reference Implementations can be deployed on a single instance of Intel® Smart Edge Open Developer Experience Kit using this feature. The Multi-Tenancy and Multi-Verticality feature is supported and tested with the following set of Reference Implementations with Developer Experience Kit v22.03: 

  • Wireless Network Ready PCB Defect Detection (v5.0.0)
  • Telehealth Remote Monitoring (v3.0.0)

NOTE: The reference implementations listed above were validated to support this feature; however, the feature is not limited to these RIs only.

 

Public Helm Registry for Helm Charts

Installation of PCB Defect Detection Reference Implementation on Intel® Smart Edge Open is accomplished using Helm charts. Earlier Helm charts used to be a part of Reference Implementation installation package. Now a global Helm repo is issued, so that Reference Implementation Helm charts can be accessible from private and public network. This will speed up and ease the process of introducing updates and their integration with the Reference Implementation.

 

Visualize the Output

To visualize the results, open Google* Chrome and navigate to the links below for respective results: 

Web Visualizer

  • http://<Controller-System-IP-Address>:30009 -DEV Mode
  • http://<Controller-System-IP-Address>:30007 -PROD Mode

Etcd UI

  • http://<Controller-System-IP-Address>:30010

To access the visualizer, log in with the username = admin and password = admin@123
 

A browser window showing the Visualizer output.
Figure 10: Application Results

 

 

 

PCB Defect Detection Results

You can view reference input and output videos of the PCB defect detection demo in GitHub. 

Refer to the PCB defects detection results section to cross-verify the results on your setup. 

Following are the reference input and output videos of PCB defect detection demo.

  • Input PCB rtsp stream: The input video file has three PCBs rotating through the screen are sent from the camera stream pod, two of these boards are defective out of three.

  • Output defect detection on PCB: The Visualizer pod showing the output of defect detection on PCB image, i.e. with red square box defect detection marked as PCB board missing component and solder bridge causing a short circuit.

 

Summary and Next Steps

The PCB demo sample application when deployed on the Intel® Smart Edge Open Developer Experience Kit creates an impactful edge computing use case that utilizes the capability of Intel® Smart Edge Open Developer Experience Kit and Open Edge Insights (OEI). 

Learn More

To continue learning, see the following guides and software resources:

Troubleshooting

 

If any issue is faced during installation, ensure that the Installation Prerequisites are configured properly. 

Pods Status Check

Verify that the pods are Ready as well as in Running state using the following command:

kubectl get pods -A

 

Pod Status shows "ImagePullBackOff" for a long time

If the pods are in ImagePullBackOff state for a long time as shown below, run the following commands.

A console window showing system output with certain pods showing "ImagePullBackOff" status.

 

Describe the pods using:

kubectl describe pod -n smartedge-apps <pod name>

 

Fetch the image name with the tag and do a manual Docker pull using the commands:

docker login
docker pull <image-name:tag>

 

Pod Status shows "ContainerCreating" for a long time

If pod status shows ContainerCreating or Error or CrashLoopBackOff for 5 minutes or more, run the following commands:

./edgesoftware uninstall -a
./edgesoftware install

 

 

Pod Status shows "pending" for a long time

If only video-analytics pod status is pending for 5 minutes or more, run the following command:

kubectl describe pods -n smartedge-apps

 

In video-analytics pod, if you see the following event due to FailedScheduling, it may be possible that AVX-512VNNI instruction set is not available in system CPU as mentioned in the NFD feature section.

Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 43s (x9 over 7m58s) default-scheduler 0/1 nodes are available: 1 node(s) didn't match Pod's node affinity/selector.

 

If all four pods etcd-ui, video-analytics, video-ingestion and web-visualizer are in pending state for more than 5 minutes, check if the Operating system is Ubuntu 20.04 LTS or not. 

 

Uninstallation Failure   

If uninstall command (./edgesoftware uninstall < module id >) fails, manually uninstall PCB Defect Detection deployment and network policy, which are specific to pcb defect detection pod: 

helm uninstall eii-deploy -n smartedge-apps; helm uninstall eii-provision -n smartedge-apps
kubectl delete networkpolicy etcd-ui-network-policy webvisualizer-network-policy -n smartedge-apps

Installation and Debug Log Info File Path 

Installation log info of Intel® Smart Edge Open Developer Experience Kit will be available at:  

/var/log/esb-cli/Smart_Edge_Open_Developer_Experience_Kits_3.0.0/Smart_Edge_Open_Developer_Experience_Kits/install.log 

 

Installation log info of Wireless Network Ready PCB Defect Detection module will be available at: 

/var/log/esb-cli/Wireless_NetworkReady_PCB_defect_detection_<version>/WirelessNetwork_Ready_PCB_defect_detection/install.log

Where <version> is the package version downloaded.

 

Execute the command below to consolidate a list of the log files in tar.gz compressed format, e.g., Wireless_NetworkReady_PCB_defect_detection.tar.gz.  

tar -czvf Wireless_NetworkReady_PCB_defect_detection.tar.gz /var/log/esb-cli/Wireless_NetworkReady_PCB_defect_detection_5.0.0/Wireless_NetworkReady_PCB_defect_detection/ /var/log/esb-cli/Smart_Edge_Open_Developer_Experience_Kits_3.0.0/Smart_Edge_Open_Developer_Experience_Kits/

 

Support Forum 

If you're unable to resolve your issues, contact the Support Forum.  

  • Make sure you have an active internet connection during the full installation. If you lose Internet connectivity at any time, the installation might fail.

  • Make sure you are using a fresh installation. Earlier software, especially Docker* and Docker Compose*, can cause issues.

  • Use the kubectl commands like describe, log, get etc. to analyze the problems.

  • Ensure the proxy settings are properly set.

  • In case of Docker pull rate limit issue, we recommend you use the Premium/PRO Docker teams to get unlimited Docker pull support.

 

 

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.