How Can a Spacecraft Use AI? Part 3 - Pose Detection

Published: 09/12/2018  

Last Updated: 09/12/2018

In our previous post, we implemented the HoG method to detect people in frames. HoG works fine when people are upright, however it does not detect people well in clusters or in unusual positions. In this tutorial, you will learn how to implement a deep neural network designed to estimate the posture of multiple people in an image using the Intel® AI DevCloud. 

HoG and SVM are a pretty powerful combination, but we can improve our detection with a pretrained neural network. We will use an open source pose estimator developed by researchers at Carnegie Mellon University and implemented for TensorFlow* and CPU by Ildoo Kim. This software enables real-time pose estimation on CPU devices thanks to Kim's selection of MobileNet*, a neural network designed for edge and mobile computing. Read more about the original implementation on GitHub*.

nasa astronaut

Intel® AI DevCloud for Post Estimation

For this tutorial, we will use the Intel AI DevCloud, but feel free to use any cloud service like AWS* or Google Cloud Platform* service for the challenge. The Intel AI DevCloud allows us to generate data much faster than with only a personal computer. 

Get started with the Intel® AI DevCloud

For users new to the Intel AI DevCloud and submitting jobs to computer clusters, take a look at the "Welcome" notebook after signing up: Browse to the following URL, replacing <userID> with your userID.

https://hub.colfaxresearch.com/user/<userID>/notebooks/Welcome.ipynb

The main techniques to learn here are:

  • Submitting a job with qsub
  • Checking the job status with qstat
  • Outputting the results with iPython magic commands

Open the notebook on the DevCloud by opening a cell at the bottom of the notebook and cloning the repository with the magic command !git clone --depth=1 --branch=demo https://github.com/JustinShenk/tf-pose-estimation

final notes

and browse to the notebook at 

https://hub.colfaxresearch.com/user/<userID>/notebooks/tf-pose-estimation/pose-estimation.ipynb

The first cell uses the magic command %%writefile to save the cell contents as pose_job.txt.

Important Remember to add a trailing newline to the cell or else the job won't run.

write file

We can call system commands in the Jupyter* Notebook by preceding a line with an exclamation mark. !ls for example calls the ls system command. And we can return the output to a variable. This allows us to check the status of the job submitted with qsub.

job = !qsub pose_job.txt
job_id = job[0].split('.')[0]

Since we want to be notified when the job is finished (a fraction of a second per image), we can use a helper function wait_for:

def wait_for(job_id):
	waiting = True
	start = time.time()
	while waiting:
		jobs = !qstat
		if any(job_id in line for line in jobs):
			print('.', end='')
			time.sleep(2) #check every 2 seconds
		else:
			end = time.time()
			duration = (end - start) / 60
			print("\nJob {} completed after {:.2f}
minutes".format(job_id, duration))
		break

Calling wait_for(job_id) prints a period every 2 seconds until the job is finished:

job files

We can then examine the output of our job (stderr, followed by stdout) and see that an image was saved at output/test.png:

github example

We can plot the contents of output with plot_dir and see our image:

pose detection

To estimate the poses for several images in a directory, replace the python src/run.py command in pose_job.txt with python src/run_directory.py --folder=<folder>.

Note Algorithms like deep neural networks can be optimized for many CPUs, FPGAs, IPUs, and VPUs (ie, Intel® Movidius™ Neural Compute Stick) on a variety of platforms by using the new Intel® OpenVINO™ toolkit.

Challenges

  • Cue lights or music in real-time with hand position (Basic)
  • Cue lights or music in real-time with gestures (Intermediate)
  • Analyze body posture over time to suggest exercise (Advanced)

​Further Challenges

  • Control music in the room by calculating optical flow
  • Optimize the real-time posture detection models to run on the Intel Movidius Neural Compute Stick  or with an Up Squared* board
  • Track the number of people in each frame of a webcam capture and use it to activate LED lights

Other Parts of these Series

How Can a Spacecraft Use Ai?

 

justin shenk

About the Author:

Justin Shenk

Master student and research assistant, University of Osnabrueck, Germany
Osnabrueck, Germany.

Justin is an AI Master thesis worker at Peltarion researching deep learning model introspection. He develops AI software as an Intel Software Innovator and demos his projects at Intel’s booths at NIPS, ICML, and CVPR. He previously worked as a neuroscientist in the US.

 

 

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.