Smart Classroom Demo - Microsoft Windows*

The Intel® Distribution of OpenVINO™ toolkit includes many different demo vision applications intended to teach developers about how to design and integrate their own applications with the toolkit. The demos span from simple image classification to human emotion detection – whatever your use-case, you can find valuable information from these demos.

One of the demos included is the Smart Classroom demo. This is a demonstration of a wider use-case for computer vision. The application uses several different neural networks to detect student actions and perform facial recognition on subjects in the classroom.

This demo can use 7 pre-trained network models:

face-detection-adas-0001: The primary detection network

landmarks-regression-retail-0009: executed on top of the face detection network to generate a vector of facial landmarks for each face

face-reidentification-retail-0095: executed on top of the face detection network to generate a vector of features for each face for the purpose of reidentification

person-detection-retail-0005: a detection network that finds people and predicts their current actions

person-detection-retail-0006: like the above, detects people and predicts their current actions. It is capable of detection more actions than person-detection-retail-0005, but is more computationally complex

person-detection-raisinghand-recognition-0001: a detection network that finds people and predicts their actions, in this case if a hand is being raised or not

person-detection-action-recognition-teacher-0002: a detection network that finds people and predicts their actions, in this case standing, writing, and demonstrating.

These models are available in the Open Model Zoo, and can be fetched using the Model Downloader available in the toolkit.

This article will walk through setting up and running the demo on Windows, using both your already available Intel® Core™ Processor and the Intel® Neural Compute Stick 2 (Intel® NCS 2). Before we begin, make sure that you meet the prerequisites.

Prerequisites

Make sure you have completed the following steps. Many of these components may have been completed during the installation of the Intel® Distribution of OpenVINO™ Toolkit, but make sure everything is installed.

• Microsoft Visual Studio* 2015/2017/2019 with C++, MSBuild, and the Build Tools for Visual Studio
  • For Visual Studio Installer 2017 and 2019, select the “Desktop development with C++” workflow
• CMake* 3.4 or higher
• At least Python* 3.6.5 64-bit with the Python libraries
  • The most recent Python3 installer from https://python.org contains all needed components. Make sure you use the 64-bit version of the installer.
• Intel® Distribution of OpenVINO™ Toolkit 2019 R2
  • Make sure that the Inference Engine Runtime for Intel® CPU and the Inference Engine Runtime for Intel® Movidius™ VPU are installed if a custom installation is desired. Otherwise, install the full package.
  • The default installation directory is C:\Program Files(x86)\IntelSWTools\openvino.

This article targets the 2019 R2 version of OpenVINO™ and uses the 2019 R2 compatible models from the Open Model Zoo. This demo should be compatible with future versions of OpenVINO™.

• Computer with Microsoft Windows® 10 64-bit OS and an Intel® Core™ processor and Integrated Webcam
  • Note that the inference engine works with AMD* processors in x86_64 environments that support AVX2 extensions, but do not have the processor-specific optimizations available with Intel® Core™ processors. An Intel® Core™ processor is recommended for use with the Intel® Distribution of OpenVINO™ Toolkit.
• Intel® Neural Compute Stick 2 (Intel® NCS 2)
  • If you’re running a current version of Windows® 10, Intel® NCS 2 works just by plugging in. If you’re using an earlier version of Windows, or the Intel® NCS 2 is not detected when running demos, then install the Movidius USB driver located in C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\inference-engine\external\MovidiusDriver\. Right-click on Movidius_VSC_Device.inf and select Install. You may need to restart your machine for changes to take effect.

Building the Demos

As long as all of the prerequisites are met, then you should continue to build the demos. The demos ship as source code, giving you the power to learn and modify for your uses. To build the demos and their Visual Studio solutions, a script has been provided in C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\open_model_zoo\demos\ named build_demos_msvc.bat. Run this in an elevated command prompt to build the demos using the command-line Build Tools for Visual Studio. If you’ve already built the
demos, you can skip this step.

After completing, the built demos and their solutions are placed in %USERPROFILE%\Documents\Intel\OpenVINO\omz_demos_build\. The primary Visual Studio solution (.sln) is located in this folder, and the individual project files are located in their respective folders. The application binaries are in intel64\Release\.

Setting Up the Environment

You’ll need to follow some steps to set up the proper environment variables and ensure that you have the right network models. Follow the instructions below:

To begin, open an elevated command prompt and scope to the OpenVINO installation directory. Run the setupvars.bat script in the /bin/ directory to set the environment variables for your current session.

cd "C:\Program Files (x86)\IntelSWTools\openvino\bin"
.\setupvars.bat

Note: You need to run this script every time you’re working in a shell. Alternatively, you can add the environment variables to your system to have them set every time a new command prompt is opened.

Fetching a Model

The models that we will be using for this demo are available in the Open Model Zoo. You can fetch these models using the Model Downloader, a script distributed with Intel® Distribution of OpenVINO™ toolkit. The Model Downloader is a Python script located at C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\tools\model_downloader\. For our purposes, we will fetch 4 models, but you can run the demo with the models listed above. The following command is an example – it fetches the models and places them in a subfolder in the same folder as the Model Downloader:

python downloader.py --name face-detection-adas-0001

python downloader.py --name landmarks-regression-retail-0009

python downloader.py --name face-reidentification-retail-0095

python downloader.py --name person-detection-action-recognition-0005

You can use the -o flag to change the output directory if desired.

You can also fetch the models directly from the Open Model Zoo at: 

https://download.01.org/opencv/2019/open_model_zoo/R2

Download the FP16 versions for inferencing on the Intel® NCS 2. Make sure you download both the .bin file and the .xml file and place them in the same folder for each model.

The Intel® Neural Compute Stick 2 requires using an FP16 model, a model that has a floating-point precision of 16 bits. FP16 models allow for inferencing with nearly the same amount of precision with less computational overhead compared to classical FP32 models. OpenVINO 2019 R2 supports the use of FP16 models with every plugin, including the MYRIAD plugin supporting the Intel® NCS 2.

Running the Demo Using Intel® NCS 2

With your model and your demo video, you’re ready to run the demo. If you’ve closed your Command Prompt before this point, you’ll need to rerun setupvars.bat in the OpenVINO installation directory to set the proper environment variables like above. After, scope to the folder that contains the demo:

cd %USERPROFILE%\Documents\Intel\OpenVINO\omz_demos_build\intel64\Release

The demos are command-line programs that use flags as options for running. The full list of options for the demo can be seen by running a demo with the –h flag:

smart_classroom_demo.exe –h

The demo can work with an active camera such as the integrated webcam in your development laptop or monitor. Make sure that you’ve plugged in your Intel® NCS 2.

smart_classroom_demo.exe –m_fd “C:\Program Files (x86)\ IntelSWTools\openvino\deployment_tools\tools\model_downloader\Transportation\object_detection\face\pruned_mobilenet_reduced_ssd_shared_weights\dldt\FP16\face-detection-adas-0001.xml” –m_lm “C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\tools\model_downloader\Retail\object_attributes\landmarks_regression\0009\dldt\FP16\landmarks-regression-retail-0009.xml” –m_reid “C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\tools\model_downloader\Retail\object_reidentification\face\mobilenet_based\dldt\FP16\face-reidentification-retail-0095.xml” –m_act “C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\tools\model_downloader\ Retail\action_detection\pedestrian\rmnet_ssd\0165\dldt\FP16\person-detection-action-recognition-0005.xml” -i cam –d_fd MYRIAD –d_lm MYRIAD –d_reid MYRIAD –d_act MYRIAD

This demo takes the output from the facial detection network and performs further inference on it using the facial landmark, facial reidentification, and action recognition networks. It runs every network on an attached Intel® NCS 2 device. For best performance, attach an Intel® NCS 2 for every network that you run simultaneously.

Inferencing Using an Intel® CPU

These demos can also be run on any computer with at least a 6th Generation Intel® Core™ Processor. You only need to change the device flag for each network. CPU is also the default plugin for any network that you don’t specify a device for.

smart_classroom_demo.exe –m_fd “C:\Program Files (x86)\ IntelSWTools\openvino\deployment_tools\tools\model_downloader\Transportation\object_detection\face\pruned_mobilenet_reduced_ssd_shared_weights\dldt\FP16\face-detection-adas-0001.xml” –m_lm “C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\tools\model_downloader\Retail\object_attributes\landmarks_regression\0009\dldt\FP16\landmarks-regression-retail-0009.xml” –m_reid “C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\tools\model_downloader\Retail\object_reidentification\face\mobilenet_based\dldt\FP16\face-reidentification-retail-0095.xml” –m_act “C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\tools\model_downloader\ Retail\action_detection\pedestrian\rmnet_ssd\0165\dldt\FP16\person-detection-action-recognition-0005.xml” -i cam –d_fd CPU –d_lm CPU –d_reid CPU –d_act CPU

You can even mix devices, splitting the compute load where necessary:

smart_classroom_demo.exe –m_fd “C:\Program Files (x86)\ IntelSWTools\openvino\deployment_tools\tools\model_downloader\Transportation\object_detection\face\pruned_mobilenet_reduced_ssd_shared_weights\dldt\FP16\face-detection-adas-0001.xml” –m_lm “C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\tools\model_downloader\Retail\object_attributes\landmarks_regression\0009\dldt\FP16\landmarks-regression-retail-0009.xml” –m_reid “C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\tools\model_downloader\Retail\object_reidentification\face\mobilenet_based\dldt\FP16\face-reidentification-retail-0095.xml” –m_act “C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\tools\model_downloader\ Retail\action_detection\pedestrian\rmnet_ssd\0165\dldt\FP16\person-detection-action-recognition-0005.xml” -i cam –d_fd CPU –d_lm MYRIAD –d_reid MYRIAD –d_act MYRIAD

We encourage you to explore the smart_classroom_demo project to see how the code interacts with the network and the Inference Engine and the best ways to integrate your application with Intel® Distribution of OpenVINO™ toolkit.