Sparse BLAS Matrix Handle Contract between User and Library
The sparse matrix handle (sparse::matrix_handle_t) takes in data from the User in a call of the form sparse::set_<format>_data(q, handle, /*user data*/), for example sparse::set_csr_data(q, handle, /*user data*/). Unlike most other oneMKL domains, the sparse::matrix_handle_t along with user data persists outside of individual calls to the oneMKL library. When the User subsequently makes a call to a oneMKL Sparse BLAS API with that handle, the Library uses that data stored in the handle internally for the various operations. Because both users and library have access to the data at the same time while the matrix handle exists, there must be some agreements about what can and cannot be done by each party. We therefore introduce an implicit contract between the User and the Library that describes each party’s roles and responsibilities with respect to the Sparse BLAS SYCL APIs and the use of the sparse::matrix_handle_t.
Description of sparse::matrix_handle_t object
First, we note that the sparse::matrix_handle_t represents a generic sparse matrix object with some internal matrix format or representation. Providing device USM pointers for that matrix data in the sparse::matrix_handle_t object binds that handle to the sycl::context and sycl::device associated with the device USM pointer and can be used on any device within the context that is compatible with that sycl::device (can read from the particular device peer to peer). Using shared and host USM pointers or sycl::buffer's for the matrix data binds that sparse::matrix_handle_t object to the associated sycl::context and could be used on any device in the context. It is the User’s responsibility to make sure they are using the matrix handle with appropriate queues and devices within the given context.
Second, the sparse::matrix_handle_t can best be described as a “view of User’s matrix data arrays with an opaque state attached to it”. That is, it is a lightweight view to begin with, but through the use of sparse::optimize_* APIs and in some cases, the execute APIs (like sparse::matmat()), the hidden state could grow with different internal optimizations/structures that can persist through the lifetime of the handle, with the goal of enabling superior performance for the desired operations.
User and Library agreements with respect to sparse::matrix_handle_t object
The following agreements describe User and Library roles with respect to the handle data ownership and use.
User agreements:
The User owns any data that is provided to the sparse::matrix_handle_t and is responsible to create and dispose of them correctly. That is, the data is created by the User then passed to the matrix handle using a Library API such as sparse::set_csr_data(). The data can be disposed of only after all uses of the matrix handle and the release of the matrix handle have finished.
The User agrees not to modify the data arrays directly while they are in a sparse::matrix_handle_t. Any changes must happen before attaching to a matrix handle or, when available, indirectly through use of a Sparse BLAS library API that states it will modify the matrix handle data in specified ways.
Library agreements:
The Library may create and store data in the sparse::matrix_handle_t during a Sparse BLAS API library call. Such data is owned by the Library, and the Library is responsible for its disposal which occurs at the respective matrix handle release. The User cannot access this data.
The Library agrees to not modify the user-provided data arrays in the sparse::matrix_handle_t unless through a library API that specifically states it may change the data (such as sparse::sort_matrix() or sparse::omatcopy()).
With the understanding of these agreements, along with understanding of the more advanced use recommendations, the User can safely use the oneMKL SYCL Sparse BLAS APIs to accelerate their workloads and applications, and the Library can rely on a consistent and known state of the user data through the lifetime of the handle.
Advanced use recommendations
The sparse::matrix_handle_t object is not currently considered thread-safe, so it must be used serially on the host.
Most of the Sparse BLAS operations consume the user-provided data in the sparse::matrix_handle in a read-only fashion. Exceptions are APIs that are filling a sparse::matrix_handle_t such as sparse::matmat() for the output C matrix (A and B are still read-only) or sparse::omatcopy() and other APIs that explicitly state they are modifying the user data, such as sparse::sort_matrix().
It is recommended for the User to not use the same data arrays in multiple sparse::matrix_handle_t's on the same context and device, but with care it may be possible to do so safely. Doing so may affect the performance.
It is also recommended for the User to not use the raw data arrays that are being used in a sparse::matrix_handle_t in a read-only fashion in other kernels or library calls independent of the sparse::matrix_handle_t on the same context and device, but with care it may be possible to do so safely. Doing so may affect the performance.
Example of sparse::matrix_handle_t usage workflow
An example workflow to demonstrate the usage model is the following:
// (A) Allocate user memory for sparse matrix such as with SYCL USM device alloc or sycl::buffer
// (B) Do anything with matrix arrays, such as memcpy some data from host
// (C) Create a sparse matrix handle and pass in matrix arrays
// (D) Make some library calls of your choice (for instance with matrix handle)
// (E) Destroy the matrix handle via call to sparse::release_matrix_handle()
// (F) Do anything with user memory - modify, copy, etc
// (G) Create another sparse matrix handle using the same arrays
// (H) Make other library calls of your choice
// (I) Destroy the matrix handle via call to sparse::release_matrix_handle()
// (J) Do whatever you want with user memory