GPU Debugging: Challenges and Opportunities

From GPU Debugging: Challenges and Opportunities presented at the International Workshop on OpenCL (IWOCL) 2017.

GPU debugging support matches the OpenCL™ 2.0 GPU/CPU driver package for Linux* (64-bit) from OpenCL™ Drivers and Runtimes for Intel® Architecture with the notable exception of processors based on the Broadwell architecture. 

Basic concepts to keep in mind for GPU debugging

• There are "host" and "target" components.  Host = where you interact with the debugger, target = where the application is run 
• There are 3 components: gdbserver, the application to be debugged, and the gdb session.
• The gdb session and application can be on the same or different machines
• Breakpoints in the graphics driver can affect screen rendering.  You cannot debug on the same system that is rendering your screen.
• However, non-graphical connections (such as with SSH) are unaffected. The "host" can be connected to remotely as well for gdb's text interface.

Abbreviations used:

KMD - Kernel Mode Driver

RT = OpenCL Runtime

DCD – Debug Companion Driver

• Ring-0 driver, provides low-level gfx access
• Run control flow, breakpoints, etc

DSL – Debug Support Library

• Ring-3 debugger driver (shared library)
• Loaded into the gdbserver process

DSL <--> DCD

• Communicate via IOCTLs

How to set up a debugging session

The simplest option is to use ssh for steps 1,2, and 3.  However, gdb can be run locally as well.  The target steps (1 and 2) should be run remotely because GPU breakpoints can cause rendering hangs.

1. launch gdbserver

/usr/bin/gdbserver-igfx :1234 --attach 123

2. launch the application

export IGFXDBG_OVERRIDE_CLIENT_PID=123
./gemm

Note: there is an automatic breakpoint at the first kernel launch

3. launch GDB

source /opt/intel/opencl-sdk/gt_debugger_2016.0/bin/debuggervars.sh

/opt/intel/opencl-sdk/gt_debugger_2016.0/bin/launch_gdb.sh –tui

In GDB

target remote :1234
continue
x/i $pc

GDB should now be able to step through the kernel code.