Tutorial: Debugging with Intel® Distribution for GDB* Using Intel® Distribution for GDB with Docker*

This document describes how to debug Docker* applications running on Intel GPU devices. Instructions how to debug natively can be found here. It describes how to create and use Docker images for the following debugging scenarios, and how to use the run image command to debug.

There are several ways to configure debugging inside containers. For now, use the following scenarios:

• Intel® Distribution for GDB and application to be debugged in a single container.

• Intel Distribution for GDB and server in one container, application to be debugged in a second container.

There are different prerequisites for each scenario mentioned above.

See Docker Engine Overview and Getting Started with Docker for more details.

A Docker container isolates a related collection of applications and their dependencies (libraries, environment variables, configuration files, scripts, etc.) from other containers and processes that are running on the same Linux* host system. However, all Docker container images running on a specific Linux host share the underlying Linux kernel and drivers. Thus, oneAPI drivers and kernel modules need to be installed and configured into the Linux system that is hosting the Docker containers for the container applications to be able to access those host machine resources. To allow debugging you must do the following:

• Install packages with the custom kernel and kernel modules.

• Set all required boot arguments to kernel. For more details see the Installation Guide

Use the array-transformexample as the target application. Compile with optimization flags -O0 -g.

Modify the example application and add following line after main():

for (int i = 1; i < argc; ++i)  if (strcmp(argv[i], "wait") == 0) getchar();

This causes our application to wait for user input if the wait parameter was passed.

Official Docker images for Intel® oneAPI Base Toolkit (Base Kit) can be found here. Alternatively, the following Dockerfile can be used:

FROM ubuntu:latest

RUN DEBIAN_FRONTEND=noninteractive \
    TZ=Europe/Berlin \
    apt-get update && \
    apt-get install -y \
    tzdata wget gpg

RUN wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB \
    | gpg --dearmor \
    | tee /usr/share/keyrings/oneapi-archive-keyring.gpg \
    > /dev/null

RUN echo "deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] \
    https://apt.repos.intel.com/oneapi all main" | \
    tee /etc/apt/sources.list.d/oneAPI.list

RUN apt-get update && \
    apt-get install -y intel-basekit

RUN echo 'source /opt/intel/oneapi/setvars.sh --force' > $HOME/.bashrc

ENV ZET_ENABLE_PROGRAM_DEBUGGING=1

NOTE: The oneAPI evironment setup shown above assumes you are using the “component” directory layout. If you wish to use the “unified” directory layout, see the oneAPI Development Environmental Setup for details.

In this basic use case, all of your development and debugging is happening in a single container. Create a Dockerfile named Dockerfile.all-in-one with content from the previous section.

Now you can build the image in the same directory where your Dockerfile is. Run the following command:

$ docker build -t intel-one-api-all-in-one -f Dockerfile.all-in-one $(pwd)

Building the image for the first time might take a longer time, but after this step you will have a Docker image called intel-one-api-all-in-one.

Then run the image and work in an isolated environment:

$ docker run \
         -it \
         --device /dev/dri \
         -v <example-source-path-on-host>:/sources \
         intel-one-api-all-in-one

The array-transformexample should now be in the /sources directory in the container. You can build the example in the container by following the build instructions in the example README file.

Before starting a debug session, we suggest running sycl-ls to see available hardware. An example output is shown below:

> $ sycl-ls
> ...
> [ext_oneapi_level_zero:gpu:0] Intel(R) Level-Zero, Intel(R) Graphics [0x56c1] 1.3 [1.3.0]
> [ext_oneapi_level_zero:gpu:1] Intel(R) Level-Zero, Intel(R) Graphics [0x56c1] 1.3 [1.3.0]
> ...

After building the application sources, you can start debugging. Run inside the container:

$ gdb-oneapi /sources/<path-to>/array-transform -ex "set args gpu" -ex "b GetDim"

Specifying -ex "set args gpu" sets the application argument to “gpu” so the test application uses the “gpu” device, which you should have seen in the sycl-ls output

The option -ex "b GetDim" puts a breakpoint in function GetDim.

Inside the Intel Distribution for GDB, you can start the debug session with (gdb) run.

In this scenario, the target application executes in a separate deployment container. The goal is to debug it remotely from a development container with gdb-oneapi.

Both containers should be in the same namespace. Assume the deployment container has the name “isolated_app_to_debug_container” (Run docker container list to get the actual name of the container with the target application running in it).

Run modified array-transform as follows:

$ docker run  -it --rm \
              --name isolated_app_to_debug_container \
              --device /dev/dri \
              -v <host-app-exec-path>:<isolated-target-app-exec-path> \
              intel-one-api-official \
              <isolated-target-app-exec-path> gpu wait

This starts the target application, which waits for an input (if you made the modifications to the example app, as described in Example application).

To start the Intel Distribution for GDB session, run the following command:

$ docker run -it \
             --cap-add SYS_ADMIN \
             --cap-add SYS_PTRACE \
             --device /dev/dri \
             --security-opt seccomp=unconfined \
             -v <sources-path-on-host>:<sources-path-in-container> \
             --pid=container:isolated_app_to_debug_container \
             --network=container:isolated_app_to_debug_container \
             intel-one-api-official \
             gdb-oneapi -p 1 -ex "dir <sources-path-in-container>" -ex "b GetDim"

After this command, Docker starts Intel Distribution for GDB, which will be attached to process 1. The previous command makes the target application reachable in the new newly created container by PID 1. Also, -ex "dir <sources-path-in-container>" specifies the directory with source files to fix file name mapping. The -ex "b GetDim" puts a breakpoint on function GetDim(). This can be skipped because breakpoints can be added later. Please note that the command starts the debugger, which attaches to and pauses the target application.

Since PID’s of containerized applications are accessible from the host machine as an optional scenario, the target application can run in the container, and the debugger can run using this command:

$ gdb-oneapi -p $(pgrep <isolated-target-app-exec-path>)``.

However, we do not recommend this approach because to get access to a target application process Linux KernelHardening should be disabled. The better solution would be to expose a port and use it in combination with gdbserver-ze.

This documentation from Intel explains how to use Intel Distribution for GDB with Docker containers based on the ubuntu:latest image. But the same approach can be used to debug in images based on other supported GNU/Linux distributions.