Prerequisites

• Install the Intel® oneAPI Base Toolkit for Windows* OS.

• Install Microsoft Visual Studio* 2019 or 2022.

  NOTE:

  Support for Visual Studio* 2017 is removed starting with the Intel® oneAPI 2022.2 release.

• Resizable BAR or Smart Access Memory must be enabled for debugging applications using Intel® Arc™ A-Series Graphics. Please follow the instructions to enable the Resizable BAR. For local GPU debugging, the Resizable BAR needs to be enabled on the host machine and for remote GPU debugging, this needs to be enabled on the target machine.

• For local GPU debugging, a Windows system with a combination of either an integrated and a discrete GPU, or multiple discrete GPUs is required. The list of supported GPU devices can be found here.

• For remote GPU debugging, two Windows systems are required: a host and a target. Visual Studio and oneAPI Base Kit must be installed on the host system. The application is deployed and run on the target system. Install the latest GPU drivers and configure your target system to use them.

• To enable Fortran debugging, install, in addition, the Fortran Compiler from the Intel® HPC Toolkit for Windows* OS.

Build the Array Transform sample (SYCL)

If you intend to debug a SYCL application, we recommend to start with the Array Transform sample.

1. In Microsoft Visual Studio, go to File > New > Browse Intel oneAPI Samples and select Debugger: Array Transform.

   If you have already fetched the sample or you have your own sample, simply open the solution file with Microsoft Visual Studio*.

2. In the Solution Explorer, right-click the array-transform project and select Properties. Alternatively, press Alt+Enter.

   1. Under Configuration Properties, select General and set Platform Toolset to Intel® oneAPI DPC++ Compiler.

   2. Select DPC++ > General and change Debug Information Format to Program Database (/Zi).

      

   3. Go to the DPC++ > Optimization tab and change Set Optimization to Disabled (/Od).

      NOTE:

      Leave Enable/Disable DPC++ early optimization before generation of SPRI-V code as default.

      

   4. Select Linker and set the Pass additional options to device compilers field to /Od. This setting disables kernel optimizations to provide a smooth debug experience.

   5. Click Apply to save the changes.

3. To build the solution, select Build > Build Solution in the main Visual Studio toolbar. In the Output window, verify that the build is successful.

Debug the host part of application locally

You are ready to run your project.

1. Set breakpoint at the line marked as success in the array-transform.cpp file.

2. From the Debug menu, select Start Debugging.

3. Click the Local Windows Debugger menu.

You will see when the thread hits the breakpoint outside the kernel. During this run the default device selector had selected the device to offload the kernel. Usually it is a L0 GPU device, when available.

Debug the host part of application remotely

Here, we assume two system setup:

• host system, where Visual Studio and oneAPI toolkits are installed;

• target system, where the application is deployed and run.

Setup the target system

1. Make sure that runtime dependencies on shared libraries from Intel® oneAPI Toolkits are available on the target system. You can do this by Installing run-time dependencies and selecting Intel® oneAPI DPC++/C++ Compiler Runtime for Windows* from the list of runtime dependencies. For Fortran remote debugging, select Intel® Fortran Compiler Runtime for Windows*.

2. Install Microsoft Visual Studio remote debugger based on the version of Visual Studio installed on your host machine. To learn more about Microsoft Visual Studio remote debugger, refer to Microsoft documentation.

Prepare a project for debugging remotely

Follow the steps below to set up the remote debugging session for the Array Transform sample:

1. Open Array Transform Properties. You need Properties window to complete all of the next steps in this section.

2. For remote debug: under Configuration Properties select Debugging.

   1. From the Debugger to launch drop-down menu, select Remote Windows* Debugger.

   2. For Remote Command, specify the path to the deployment directory on the target system. For example, C:\deploy\array-transform.exe.

   3. Set Working Directory to the path to the deployment directory on the target system.

   4. Update Remote Server Name using the remote debugger name from the target system, including the port number. The default port used by the Microsoft Visual Studio 2022 Remote Debugger server is 4026.

      NOTE:

      You can get the Remote Server name from the window that appears on the target system when you start the Remote Debugger.

   5. Set Deployment Directory to the path to the deployment directory on the target system.

      The directory will be created on the target system and the array-transform executable file will be copied to this location.

      NOTE:

      Remote Command, Working Directory, and Deployment Directory fields must point to the same path.
   

3. Click the Apply button to set your changes.

4. Click the Configuration Manager in the upper right corner of the Properties window and mark the Deploy check box. Close the Property Pages.

   NOTE:

   The Deploy check box is disabled until the Deployment Directory is specified.

   

Debug the sample with the breakpoint outside the kernel

To verify the remote debugger setup, start the debugging with the breakpoint outside the kernel:

1. On the target system, start the remote debugger by clicking the Remote Debugger in the Start menu.

   

2. On the host system, return to your project and set a breakpoint outside the kernel offloaded to the GPU. For example, at line marked as success of the array-transform.cpp file.

3. To start debugging, on the Debug toolbar, click Remote Windows Debugger button.

   NOTE:

   You may be prompted to log in to the target system. Be sure that credentials are set for the target system.

   When the debugger hits the breakpoint, the example Visual Studio output is as follows:

   

   Your application will launch on the target system. The offloading will happen to the default device or the device specified by ONEAPI_DEVICE_SELECTOR. The example output of the Remote Windows Debugger on the target system is as follows:

   [SYCL] Using device: [Intel(R) Arc(TM) A770 Graphics] from [Intel(R) Level-Zero]

   Such output indicates that you set up the remote debugger successfully and the default L0 GPU device was used for offloading.

4. Quit the debugger by clicking the button.

Get Started with Debugging a Kernel on CPU

Configure application to offload on CPU

In the Solution Explorer, right-click the array-transform project and select Properties. Alternatively, press Alt+Enter.

Under Configuration Properties, select Debugging. For the debugger you intend to use (Local Windows Debugger or Remote Windows Debugger), edit the Environment field such that the value of the ONEAPI_DEVICE_SELECTOR environment variable is set to *:cpu.

For more details about the device selection, please refer to Compiler Environment Variables.

Debug kernel offloaded to CPU locally

You are ready to debug your project.

1. Set breakpoint at line marked as breakpoint-here in the array-transform.cpp file.

2. From the Debug menu, select Start Debugging.

3. Click the Local Windows Debugger menu.

You will see when the thread hits the breakpoint.

Refer to the tutorial to learn more about using Intel Distribution for GDB.

Debug kernel offloaded to CPU remotely

By now, if you tried to set a breakpoint inside the kernel offloaded to CPU and used the Remote Windows Debugger, the breakpoint was not hit. The reason is that the kernel’s .PDB file is located on the remote machine and Visual Studio cannot find it. To fix that additionally to the settings described in Debug the Host Part of Application Remotely, do the following:

1. On target system make the directory %USERPROFILE%\AppData\Local\Temp accessible by Visual Studio, such that it could load kernel debug symbols.

   1. On the target system, go to %USERPROFILE%\AppData\Local\Temp and open its properties.

   2. Go to the Sharing tab and click Share. A new window pops up. By default, the sharing is configured for the current user.

   3. Click Share button on the bottom of the window to share the folder for the selected users.

   4. Copy the network path of the shared folder.

2. Check that the folder is accessible on your host system by its network path.

3. When the folder is available, on the host machine in Visual Studio, go to Tools > Options > Debugging > Symbols. Click a plus-icon to add a new location, from where Visual Studio loads debug symbols, and type the network path of the shared folder there.

With the above setting, the breakpoint inside the kernel offloaded to a CPU device should be hit with the Remote Windows Debugger.

Get started with debugging a kernel on GPU

High-level steps

1. Choose between local and remote debugging and hit a breakpoint outside of the kernel.

2. Configure target system.

3. Prepare an application for debugging using the oneAPI plugin of Microsoft Visual Studio.

4. Configure Intel Distribution for GDB.

5. Start the debugging session, which will deploy and run the application on the target system.

6. Investigate the program state using standard Visual Studio debugging user interface. Additionally, Intel Distribution for GDB provides the oneAPI GPU threads view.

Choose between local and remote debugging

Debugging the kernel offloaded to a GPU device halts the GPU device on which the kernel is offloaded. If the display is also using the same GPU, it will be frozen. When you have multiple GPUs on your system, you can debug locally. Typically, one GPU (often an integrated one) is dedicated to displaying, while the remaining GPUs can be utilized for computation and debugging if they are supported by the debugger. In that case you can use the local debugger.

If your target system, where you plan to run the application offloading to a GPU, has just one GPU device and this GPU is supported by the debugger, you can debug the kernel only with the remote debugger. The high-level setup of the remote system looks as following:

Before you proceed, please check that you can hit a breakoint outside of the kernel with the choosen debugger:

• the local debugger;

• the remote debugger.

Configure the target system

For debugging kernels offloaded to GPU, Intel Distribution for GDB target installer (gen_debugger_target.msi) has to be installed on the machine, where the application will run.

1. [only for remote debugging] If you intend to debug remotely, copy the target installer of Intel Distribution for GDB (gen_debugger_target.msi) from the host system to the target.

   The installer is located on the host machine at %ProgramFiles(x86)%\Intel\oneAPI\debugger\latest\opt\debugger\target.

2. Run the installer on host for local debugging or on target for the remote debugging. While installing, select the checkbox under “Additional Installer Task” to set the TdrDelay to the default value of 300 seconds.

   NOTE:

   The installer sets the following registry keys to 300 seconds:

   • HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\GraphicsDrivers\TdrDelay

   This setting specifies the number of seconds that the GPU can delay the preempt request from the GPU scheduler. This is effectively the timeout threshold.

   Without this setting, the debugger will not work properly. The stop at a breakpoint triggers the timeout detection and recovery (TDR) mechanism on Windows, which we need to delay in order to investigate the state where the debugger has stopped the kernel.

   OS will not be able to recover automatically if the GPU stops responding while producing a graphics output. Ideally, install the remote debugger only on computing GPUs.

Set Up Intel Distribution for GDB with enhanced GPU offload

In Visual Studio configure Intel(R) Distribution for GDB* extension.

1. Open Tools > Options > Intel® oneAPI > Intel® Distribution for GDB*.

2. Set Enable Debugging to True.

3. (Optional for Remote Debugging) Enable secure connection to the remote target by setting Enable SSH SSL Tunnel to True.

   If this option is enabled, you will be asked to enter user credentials while connecting to the remote target system. You can also change the Maximum Number of Retries to control how many times a user can re-enter credentials.

   NOTE:

   Security of the SSH configuration defines security of your system. See the NIST SSH guidelines on setting a strong password.

   

4. Enter the oneAPI base toolkit install directory that was chosen during installation. Leave this field blank if the oneAPI base toolkit was installed in the default location.

5. Set TCP/IP port to the port number used when running gdbserver-ze. The default is 1234.

   NOTE:

   The TCP/IP port must differ from the port number used by Microsoft Visual Studio remote debugger.
   

6. Click OK to save the changes.

Configure and debug the application

The further setup procedure depends on whether you want to debug a program following SYCL or OpenMP™ standards. For remote debugging, configure your systems as described in the following sections:

• SYCL Application

• OpenMP Application

Prepare to debug a SYCL kernel on a GPU

1. If not yet done, configure and build the application.

2. Go to Project Property pages and under Configuration Properties select Debugging.

   1. From the Debugger to launch drop-down menu, select the debugger you wish to use: Local Windows* Debugger or Remote Windows* Debugger.

   2. Edit the Environment field such that the value of the ONEAPI_DEVICE_SELECTOR environment variable is set to level_zero:gpu. Also, add the environment variable ZET_ENABLE_PROGRAM_DEBUGGING and set it’s value to 1.

      NOTE:

      When there are multiple GPUs on the target machine you are debugging, you need to find the device number of the GPU using the commmand sycl-ls in the command prompt of the target. Since debugging is supported only on Intel® Arc™ Series GPUs, make sure you select the device number associated with these discrete GPUs.

      Once you have identified the device number, you will have to specify this in the Environment field such that the value of the ONEAPI_DEVICE_SELECTOR environment variable is set to level_zero:<device_number>.

Prepare to debug an OpenMP kernel on a GPU

To debug an OpenMP project, set environment variables as follows:

• Set device type to GPU:

  LIBOMPTARGET_DEVICETYPE=GPU

• Choose level0 as an offload target:

  LIBOMPTARGET_PLUGIN=level0

• Configure flags for the backend compiler.

  For level0:

  LIBOMPTARGET_LEVEL0_COMPILATION_OPTIONS=-g -cl-opt-disable

• (Optional) Print offload profile information at the end of offload:

  LIBOMPTARGET_PROFILE=T

• (Optional) Print OpenMP library debug traces:

  LIBOMPTARGET_DEBUG=1

• Set the environment variable to enable program debugging:

  ZET_ENABLE_PROGRAM_DEBUGGING=1

To prepare a project for debugging, do the following:

1. Create a new SYCL Console App project.

2. Set Property Pages > General > Platform Toolset to Intel(R) oneAPI DPC++ Compiler 2023. Press Apply.

3. Set Property Pages > Debugging > Environment as shown above.

4. Set Property Pages > DPC++ > Language > OpenMP Support to Generate Parallel Code (/Qiopenmp).

5. Set Property Pages > DPC++ > Language > Enable OpenMP Offloading to Generate x86 + SPIR64 fat binary (/Qopenmp-targets:spir64).

6. Add your code and compile the application.

To prepare a Fortran project for debugging, do the following:

1. Make sure you have installed:

   1. on the host system, the Fortran Compiler from the Intel® HPC Toolkit for Windows* OS.

   2. on the target system, the Intel® Fortran Compiler Runtime for Windows* from the Intel® oneAPI standalone component installation files.

2. Create a new Fortran project.

3. Set Property Pages > General > Use Compiler to IFX Intel(R) Fortran Compiler. Press Apply.

4. Set Property Pages > Debugging > Environment as shown above.

5. Set Property Pages > Fortran > Code Generation > Enable OpenMP Offloading to Generate x86 + SPIR64 fat binary (/Qopenmp-targets:spir64).

6. Set Property Pages > Fortran > Language > Process OpenMP Directives to Generate Parallel Code (/Qopenmp).

7. Set Property Pages > Fortran > Language > OpenMP Support to Generate Parallel Code (/Qiopenmp).

8. Add your code and compile the application.

9. Copy your executable to the deployment directory on your target system.

10. Under Property Pages select Debugging.

    1. From the Connection drop-down menu, select Remote with Windows authentication

    2. Update Remote Server Name using the remote debugger name from the target system, including the port number. The default port used by the Microsoft Visual Studio 2022 Remote Debugger server is 4026.

    3. For Remote Command, specify the path to the deployment directory on the target system. For example, C:\deploy\local_vars.exe.

Build and debug the sample

To build the solution, select Build > Build Solution in the main Visual Studio toolbar. In the Output window, verify that the build is successful.

Now you are ready to debug your project with the kernel offloaded to the GPU. Follow the steps in the tutorial to start the debugging.