How it Works
The Edge Insights for Autonomous Mobile Robots (EI for AMR) modules are
deployed via Docker* containers for enhanced Developer Experience (DX),
support of Continuous Integration and Continuous Deployment (CI/CD)
practices and flexible deployment in different execution environments,
including robot, development PC, Edge Server, and Cloud.
This section provides an overview of the modules and services featured
with Edge Insights for Autonomous Mobile Robots.
Modules and Services
The middleware layered architecture in the Intel® oneAPI Base Toolkit
(Base Kit) and Intel® Distribution of OpenVINO™ toolkit (OpenVINO™)
abstracts hardware dependencies from the algorithm implementation.
The ROS 2 with data distribution service (DDS) is used as a message bus.
This Publisher-Subscriber architecture based on ROS 2 topics decouples
data providers from consumers.
Camera and LIDAR sensor data is abstracted with ROS 2 topics.
Video streaming processing pipelines are supported by GStreamer*. It
decouples sensor ingestion, video processing and AI object detection via
OpenVINO™ toolkit DL Streamer framework.
Also, more complex computational graphs that decouple Sense-Plan-Act
autonomous mobile robot applications can be implemented using ROS 2 topic
registration.
This diagram shows the software components included in the EI for AMR
package. The software stack will keep evolving iteratively with
additional algorithms, applications, and third-party ecosystem software
components. For ease of use, the complete software stack is provided in
a single container called
Full Flavour container
. This container is
constructed hierarchically by extending the OpenVINO container
,
which itself extends the ROS2 container
. For storage space savings,
you can choose to run just the ROS2 container
, or the
OpenVINO container
depending on the needs of your applications.- TheROS2 containerincludes the ROS 2 middleware and tools, Intel® RealSense™ SDK and ROS 2 wrapper, GStreamer* and build tools, ROS 2 packages (Cartographer, Navigation, RTAB_MAP) and the Fast Mapping application (the Intel-optimized version of octomap).
- TheOpenVINO containerincludes theROS2 container, as well as the OpenVINO™ development toolkit, the OpenVINO™ DL GStreamer* plugins and the Wandering demonstration application.
- The AMR Full Flavour container includes the OpenVINO™ container, as well as the Intel® oneAPI Base Toolkit, the Data Parallel C++ (DPC++) compatibility tool and profiler, analyzer tools.

The EI for AMR software stack is based on software supported by and part
of the underlying hardware platform, their respective Unified Extensible
Firmware Interface (UEFI) based boot, and their supported Linux*
operating system. For details, see
Requirements.
Drivers
Edge Insights for Autonomous Mobile Robots relies on standard Intel®
Architecture Linux* drivers included and upstreamed in the Linux* kernel
from kernel.org and included in Ubuntu* distributions. These drivers
are not included in the EI for AMR package. Examples include Wi-Fi*,
Ethernet, Integrated GPU, USB, PCIe*, and others.
Middleware
EI for AMR integrates the following middleware packages:
- Robot Operating System (ROS) software stack V2 (ROS 2) from the open source project (ros.org), able to execute sample ROS 2 tutorial applications
- The Robot Operating System (ROS) is a set of open source software libraries and tools for building robot applications.
- ROS 2 depends on other middleware, like the Object Management Group (OMG) DDS connectivity framework that uses a publish–subscribe pattern.
- Intel® Distribution of OpenVINO™ toolkit 2021.2, supporting heterogeneous execution across an Intel CPU, integrated graphics, Intel® Neural Compute Stick 2 (Intel® NCS2), and Intel® Vision Accelerator Design with Intel® Movidius™ Myriad™ VPUs
- Intel® oneAPI Base Toolkit, able to execute Intel® oneAPI Base Toolkit sample applications. The Intel® oneAPI Base Toolkit is a core set of tools and libraries for developing high-performance, data-centric applications across diverse architectures. It features an industry-leading C++ compiler and the Data Parallel C++ (DPC++) language, an evolution of C++ for heterogeneous computing.For Intel® oneAPI Base Toolkit training and a presentation of the CUDA converter, refer to:
- RealSense SDK and RealSense ROS 2 Wrapper, able to execute Intel® RealSense™ ROS 2 Wrapper sample applications
- GStreamer* including support for libv4l2 video sources and GStreamer* plugins
Algorithms
Edge Insights for Autonomous Mobile Robots includes reference algorithms
as well as deep learning models as working examples for the following
automated robot control functional areas:
- ROS2 Cartographer 2D LIDAR SLAM: A cartographer is a system that provides real-time SLAM, here based on real-time 2D LIDAR sensor data. It is used to generate as-built floor plans or maps.
- ROS 2 Intel® RealSense™ vSLAM: Visual Simultaneous Localization and Mapping (vSLAM) can map the location and create a 3D virtual map, based on computer vision.
- ROS 2 Intel® RealSense™ FastMapping Algorithm: An algorithm on top of ROS that generates and maintains a volumetric representation of the environment given the depth image input of Intel® RealSense™ camera (e.g. D435i) and the position from a localization algorithm, such as General SLAM (GSLAM), RTAB_MAP, vSLAM, etc.
- ROS2 Navigation: The ROS 2 Nav2 Navigation Stack seeks to find a safe way to have a mobile robot move from point A to point B. This will complete dynamic path planning, compute velocities for motors, detect and avoid obstacles, and structure recovery behaviors.
- OpenVINO™ Model Zoo: Optimized deep learning models and a set of demos expedite development of high-performance deep learning inference applications. You can use these pre-trained models instead of training your own models to speed up the development and production deployment process.
Applications
- Wandering AI Application: The Intel® RealSense™ AI Wandering ROS 2 sample application demonstrates the combination of the middleware, algorithms, and the ROS 2 navigation stack to move a robot around a room avoiding hitting obstacles, updating a local map in real time exposed as ROS topic, and publish AI-based objects detected in another ROS topic. It uses the robot’s sensors and actuators that are available from the robot’s hardware configuration.
Tools
ROS Tools
Edge Insights for Autonomous Mobile Robots is validated using ROS 2 nodes. ROS 1 is not compatible with EI for AMR
components. A ROS 1 bridge is included to allow EI for AMR components to
interface with ROS 1 components.
- From the hardware perspective of the supported platforms, there are no known limitations for ROS 1 components.
- For information on porting ROS 1 applications to ROS 2, here is a guide from the ROS community.
Edge Insights for Autonomous Mobile Robots includes:
- rviz2, to visualize ROS topics
- rqt, a software framework of ROS that implements the various Graphical User Interface (GUI) tools in the form of plugins
- colcon, a collective construction command line tool to improve the workflow of building, testing, and using multiple software packages. It automates the process, handles the ordering, and sets up the environment to use the packages.
Simulation
The Edge Insights for Autonomous Mobile Robots includes:
- The Gazebo* robot simulator, making it possible to rapidly test algorithms, design robots, perform regression testing, and train AI systems using realistic scenarios. Gazebo offers the ability to simulate populations of robots accurately and efficiently in complex indoor and outdoor environments.
- An industrial simulation room model for Gazebo*, the Open Source Robotics Foundation (OSRF) Gazebo Environment for Agile Robotics (GEAR) workcell that was used for the ARIAC competition in 2018
Terminology
The table below lists terminology used in this document.
Term | Description |
---|---|
ARIAC | Agile Robotics for Industrial Automation Competition |
CNDA | Corporate Non-Disclosure Agreement |
CPU | Central Processing Unit |
DPC++ | Data Parallel C++ |
DL | Deep Learning |
EI for AMR | Edge Insights for Autonomous Mobile Robots |
EOF | end-of-file |
FM | Fast Mapping |
GEAR | Gazebo Environment for Agile Robotics |
GPU | Graphics Processor Unit |
GSLAM | General Simultaneous Localization and Mapping |
GUI | Graphical User Interface |
IDE | Integrated Development Environment |
IE | Inference Engine |
IPU | Image Processing Unit |
KVM | Kernel-based Virtual Machine |
LIDAR | Light Detection and Ranging |
NN | Neural Network |
OSRF | Open Source Robotics Foundation |
RDC | Resource and Documentation Center |
ROS | Robot Operating System |
RPM | Red Hat* Package Manager |
SDK | Software Development Kit |
SLAM | Simultaneous Localization And Mapping |
SSD | Single-Shot multibox Detection |
UEFI | Unified Extensible Firmware Interface |
vSLAM | Visual Simultaneous Localization and Mapping |