Get Started Guide

  • 2021.3
  • 10/21/2021
  • Public Content

How it Works

The Edge Insights for Autonomous Mobile Robots (EI for AMR) modules are deployed via Docker* containers for enhanced Developer Experience (DX), support of Continuous Integration and Continuous Deployment (CI/CD) practices and flexible deployment in different execution environments, including robot, development PC, Edge Server, and Cloud.
This section provides an overview of the modules and services featured with Edge Insights for Autonomous Mobile Robots.

Modules and Services

The middleware layered architecture in the Intel® oneAPI Base Toolkit (Base Kit) and Intel® Distribution of OpenVINO™ toolkit (OpenVINO™) abstracts hardware dependencies from the algorithm implementation.
The ROS 2 with data distribution service (DDS) is used as a message bus. This Publisher-Subscriber architecture based on ROS 2 topics decouples data providers from consumers.
Camera and LIDAR sensor data is abstracted with ROS 2 topics.
Video streaming processing pipelines are supported by GStreamer*. It decouples sensor ingestion, video processing and AI object detection via OpenVINO™ toolkit DL Streamer framework.
Also, more complex computational graphs that decouple Sense-Plan-Act autonomous mobile robot applications can be implemented using ROS 2 topic registration.
This diagram shows the software components included in the EI for AMR package. The software stack will keep evolving iteratively with additional algorithms, applications, and third-party ecosystem software components. For ease of use, the complete software stack is provided in a single container called
Full Flavour container
. This container is constructed hierarchically by extending the
OpenVINO container
, which itself extends the
ROS2 container
. For storage space savings, you can choose to run just the
ROS2 container
, or the
OpenVINO container
depending on the needs of your applications.
  • The
    ROS2 container
    includes the ROS 2 middleware and tools, Intel® RealSense™ SDK and ROS 2 wrapper, GStreamer* and build tools, ROS 2 packages (Cartographer, Navigation, RTAB_MAP) and the Fast Mapping application (the Intel-optimized version of octomap).
  • The
    OpenVINO container
    includes the
    ROS2 container
    , as well as the OpenVINO™ development toolkit, the OpenVINO™ DL GStreamer* plugins and the Wandering demonstration application.
  • The AMR Full Flavour container includes the OpenVINO™ container, as well as the Intel® oneAPI Base Toolkit, the Data Parallel C++ (DPC++) compatibility tool and profiler, analyzer tools.
The EI for AMR software stack is based on software supported by and part of the underlying hardware platform, their respective Unified Extensible Firmware Interface (UEFI) based boot, and their supported Linux* operating system. For details, see Requirements.

Drivers

Edge Insights for Autonomous Mobile Robots relies on standard Intel® Architecture Linux* drivers included and upstreamed in the Linux* kernel from kernel.org and included in Ubuntu* distributions. These drivers are not included in the EI for AMR package. Examples include Wi-Fi*, Ethernet, Integrated GPU, USB, PCIe*, and others.

Middleware

EI for AMR integrates the following middleware packages:
  • Robot Operating System (ROS) software stack V2 (ROS 2) from the open source project (ros.org), able to execute sample ROS 2 tutorial applications
    • The Robot Operating System (ROS) is a set of open source software libraries and tools for building robot applications.
    • ROS 2 depends on other middleware, like the Object Management Group (OMG) DDS connectivity framework that uses a publish–subscribe pattern.
  • Intel® Distribution of OpenVINO™ toolkit 2021.2, supporting heterogeneous execution across an Intel CPU, integrated graphics, Intel® Neural Compute Stick 2 (Intel® NCS2), and Intel® Vision Accelerator Design with Intel® Movidius™ Myriad™ VPUs
  • Intel® oneAPI Base Toolkit, able to execute Intel® oneAPI Base Toolkit sample applications. The Intel® oneAPI Base Toolkit is a core set of tools and libraries for developing high-performance, data-centric applications across diverse architectures. It features an industry-leading C++ compiler and the Data Parallel C++ (DPC++) language, an evolution of C++ for heterogeneous computing.
    For Intel® oneAPI Base Toolkit training and a presentation of the CUDA converter, refer to:
  • RealSense SDK and RealSense ROS 2 Wrapper, able to execute Intel® RealSense™ ROS 2 Wrapper sample applications
  • GStreamer* including support for libv4l2 video sources and GStreamer* plugins

Algorithms

Edge Insights for Autonomous Mobile Robots includes reference algorithms as well as deep learning models as working examples for the following automated robot control functional areas:
  • ROS2 Cartographer 2D LIDAR SLAM: A cartographer is a system that provides real-time SLAM, here based on real-time 2D LIDAR sensor data. It is used to generate as-built floor plans or maps.
  • ROS 2 Intel® RealSense™ vSLAM: Visual Simultaneous Localization and Mapping (vSLAM) can map the location and create a 3D virtual map, based on computer vision.
  • ROS 2 Intel® RealSense™ FastMapping Algorithm: An algorithm on top of ROS that generates and maintains a volumetric representation of the environment given the depth image input of Intel® RealSense™ camera (e.g. D435i) and the position from a localization algorithm, such as General SLAM (GSLAM), RTAB_MAP, vSLAM, etc.
  • ROS2 Navigation: The ROS 2 Nav2 Navigation Stack seeks to find a safe way to have a mobile robot move from point A to point B. This will complete dynamic path planning, compute velocities for motors, detect and avoid obstacles, and structure recovery behaviors.
  • OpenVINO™ Model Zoo: Optimized deep learning models and a set of demos expedite development of high-performance deep learning inference applications. You can use these pre-trained models instead of training your own models to speed up the development and production deployment process.

Applications

  • Wandering AI Application: The Intel® RealSense™ AI Wandering ROS 2 sample application demonstrates the combination of the middleware, algorithms, and the ROS 2 navigation stack to move a robot around a room avoiding hitting obstacles, updating a local map in real time exposed as ROS topic, and publish AI-based objects detected in another ROS topic. It uses the robot’s sensors and actuators that are available from the robot’s hardware configuration.

Tools

ROS Tools
Edge Insights for Autonomous Mobile Robots is validated using ROS 2 nodes. ROS 1 is not compatible with EI for AMR components. A ROS 1 bridge is included to allow EI for AMR components to interface with ROS 1 components.
  • From the hardware perspective of the supported platforms, there are no known limitations for ROS 1 components.
  • For information on porting ROS 1 applications to ROS 2, here is a guide from the ROS community.
Edge Insights for Autonomous Mobile Robots includes:
  • rviz2, to visualize ROS topics
  • rqt, a software framework of ROS that implements the various Graphical User Interface (GUI) tools in the form of plugins
  • colcon, a collective construction command line tool to improve the workflow of building, testing, and using multiple software packages. It automates the process, handles the ordering, and sets up the environment to use the packages.
Simulation
The Edge Insights for Autonomous Mobile Robots includes:
  • The Gazebo* robot simulator, making it possible to rapidly test algorithms, design robots, perform regression testing, and train AI systems using realistic scenarios. Gazebo offers the ability to simulate populations of robots accurately and efficiently in complex indoor and outdoor environments.
  • An industrial simulation room model for Gazebo*, the Open Source Robotics Foundation (OSRF) Gazebo Environment for Agile Robotics (GEAR) workcell that was used for the ARIAC competition in 2018

Terminology

The table below lists terminology used in this document.
Term
Description
ARIAC
Agile Robotics for Industrial Automation Competition
CNDA
Corporate Non-Disclosure Agreement
CPU
Central Processing Unit
DPC++
Data Parallel C++
DL
Deep Learning
EI for AMR
Edge Insights for Autonomous Mobile Robots
EOF
end-of-file
FM
Fast Mapping
GEAR
Gazebo Environment for Agile Robotics
GPU
Graphics Processor Unit
GSLAM
General Simultaneous Localization and Mapping
GUI
Graphical User Interface
IDE
Integrated Development Environment
IE
Inference Engine
IPU
Image Processing Unit
KVM
Kernel-based Virtual Machine
LIDAR
Light Detection and Ranging
NN
Neural Network
OSRF
Open Source Robotics Foundation
RDC
Resource and Documentation Center
ROS
Robot Operating System
RPM
Red Hat* Package Manager
SDK
Software Development Kit
SLAM
Simultaneous Localization And Mapping
SSD
Single-Shot multibox Detection
UEFI
Unified Extensible Firmware Interface
vSLAM
Visual Simultaneous Localization and Mapping

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.