Deferred Host Operations

Certain Vulkan commands are inherently expensive for the host CPU to execute. It is often desirable to offload such work onto background threads, and to parallelize the work across multiple CPUs. The concept of deferred operations allows applications and drivers to coordinate the execution of expensive host commands using an application-managed thread pool.

The VK_KHR_deferred_host_operations extension defines the infrastructure and usage patterns for deferrable commands, but does not specify any commands as deferrable. This is left to additional dependent extensions. Commands must not be deferred unless the deferral is specifically allowed by another extension which depends on VK_KHR_deferred_host_operations. This specification will refer to such extensions as deferral extensions.

Requesting Deferral

When an application requests an operation deferral, the implementation may defer the operation. When deferral is requested and the implementation defers any operation, the implementation must return VK_OPERATION_DEFERRED_KHR as the success code if no errors occurred. When deferral is requested, the implementation should defer the operation when the workload is significant, however if the implementation chooses not to defer any of the requested operations and instead executes all of them immediately, the implementation must return VK_OPERATION_NOT_DEFERRED_KHR as the success code if no errors occurred.

A deferred operation is created complete with an initial result value of VK_SUCCESS. The deferred operation becomes pending when an operation has been successfully deferred with that deferred operation object.

A deferred operation is considered pending until the deferred operation completes. A pending deferred operation becomes complete when it has been fully executed by one or more threads. Pending deferred operations will never complete until they are joined by an application thread, using vkDeferredOperationJoinKHR. Applications can join multiple threads to the same deferred operation, enabling concurrent execution of subtasks within that operation.

The application can query the status of a VkDeferredOperationKHR using the vkGetDeferredOperationMaxConcurrencyKHR or vkGetDeferredOperationResultKHR commands.

Parameters to the command requesting a deferred operation may be accessed by the implementation at any time until the deferred operation enters the complete state. The application must obey the following rules while a deferred operation is pending:

  • Externally synchronized parameters must not be accessed.

  • Pointer parameters must not be modified (e.g. reallocated/freed).

  • The contents of pointer parameters which may be read by the command must not be modified.

  • The contents of pointer parameters which may be written by the command must not be read.

  • Vulkan object parameters must not be passed as externally synchronized parameters to any other command.

When the deferred operation is complete, the application should call vkGetDeferredOperationResultKHR to obtain the VkResult indicating success or failure of the operation. The VkResult value returned will be one of the values that the command requesting the deferred operation is able to return. Writes to output parameters of the requesting command will happen-before the deferred operation is complete.

When a deferral is requested for a command, the implementation may perform memory management operations on the allocator supplied to vkCreateDeferredOperationKHR for the deferred operation object, as described in the Memory Allocation chapter. Such allocations must occur on the thread which requests deferral.

If an allocator was supplied for the deferred command at the time of the deferral request, then the implementation may perform memory management operations on this allocator during the execution of vkDeferredOperationJoinKHR. These operations may occur concurrently and may be performed by any joined thread. The application must ensure that the supplied allocator is able to operate correctly under these conditions.

Deferred Host Operations API

The VkDeferredOperationKHR handle is defined as:

// Provided by VK_KHR_deferred_host_operations
VK_DEFINE_NON_DISPATCHABLE_HANDLE(VkDeferredOperationKHR)

This handle refers to a tracking structure which manages the execution state for a deferred command.

To construct the tracking object for a deferred command, call:

// Provided by VK_KHR_deferred_host_operations
VkResult vkCreateDeferredOperationKHR(
    VkDevice                                    device,
    const VkAllocationCallbacks*                pAllocator,
    VkDeferredOperationKHR*                     pDeferredOperation);
  • device is the device which owns operation.

  • pAllocator controls host memory allocation as described in the Memory Allocation chapter.

  • pDeferredOperation is a pointer to a handle in which the created VkDeferredOperationKHR is returned.

Valid Usage (Implicit)
  • VUID-vkCreateDeferredOperationKHR-device-parameter
    device must be a valid VkDevice handle

  • VUID-vkCreateDeferredOperationKHR-pAllocator-parameter
    If pAllocator is not NULL, pAllocator must be a valid pointer to a valid VkAllocationCallbacks structure

  • VUID-vkCreateDeferredOperationKHR-pDeferredOperation-parameter
    pDeferredOperation must be a valid pointer to a VkDeferredOperationKHR handle

Return Codes
Success
  • VK_SUCCESS

Failure
  • VK_ERROR_OUT_OF_HOST_MEMORY

To assign a thread to a deferred operation, call:

// Provided by VK_KHR_deferred_host_operations
VkResult vkDeferredOperationJoinKHR(
    VkDevice                                    device,
    VkDeferredOperationKHR                      operation);
  • device is the device which owns operation.

  • operation is the deferred operation that the calling thread should work on.

The vkDeferredOperationJoinKHR command will execute a portion of the deferred operation on the calling thread.

The return value will be one of the following:

  • A return value of VK_SUCCESS indicates that operation is complete. The application should use vkGetDeferredOperationResultKHR to retrieve the result of operation.

  • A return value of VK_THREAD_DONE_KHR indicates that the deferred operation is not complete, but there is no work remaining to assign to threads. Future calls to vkDeferredOperationJoinKHR are not necessary and will simply harm performance. This situation may occur when other threads executing vkDeferredOperationJoinKHR are about to complete operation, and the implementation is unable to partition the workload any further.

  • A return value of VK_THREAD_IDLE_KHR indicates that the deferred operation is not complete, and there is no work for the thread to do at the time of the call. This situation may occur if the operation encounters a temporary reduction in parallelism. By returning VK_THREAD_IDLE_KHR, the implementation is signaling that it expects that more opportunities for parallelism will emerge as execution progresses, and that future calls to vkDeferredOperationJoinKHR can be beneficial. In the meantime, the application can perform other work on the calling thread.

Implementations must guarantee forward progress by enforcing the following invariants:

  1. If only one thread has invoked vkDeferredOperationJoinKHR on a given operation, that thread must execute the operation to completion and return VK_SUCCESS.

  2. If multiple threads have concurrently invoked vkDeferredOperationJoinKHR on the same operation, then at least one of them must complete the operation and return VK_SUCCESS.

Valid Usage (Implicit)
  • VUID-vkDeferredOperationJoinKHR-device-parameter
    device must be a valid VkDevice handle

  • VUID-vkDeferredOperationJoinKHR-operation-parameter
    operation must be a valid VkDeferredOperationKHR handle

  • VUID-vkDeferredOperationJoinKHR-operation-parent
    operation must have been created, allocated, or retrieved from device

Return Codes
Success
  • VK_SUCCESS

  • VK_THREAD_DONE_KHR

  • VK_THREAD_IDLE_KHR

Failure
  • VK_ERROR_OUT_OF_HOST_MEMORY

  • VK_ERROR_OUT_OF_DEVICE_MEMORY

When a deferred operation is completed, the application can destroy the tracking object by calling:

// Provided by VK_KHR_deferred_host_operations
void vkDestroyDeferredOperationKHR(
    VkDevice                                    device,
    VkDeferredOperationKHR                      operation,
    const VkAllocationCallbacks*                pAllocator);
  • device is the device which owns operation.

  • operation is the completed operation to be destroyed.

  • pAllocator controls host memory allocation as described in the Memory Allocation chapter.

Valid Usage
  • VUID-vkDestroyDeferredOperationKHR-operation-03434
    If VkAllocationCallbacks were provided when operation was created, a compatible set of callbacks must be provided here

  • VUID-vkDestroyDeferredOperationKHR-operation-03435
    If no VkAllocationCallbacks were provided when operation was created, pAllocator must be NULL

  • VUID-vkDestroyDeferredOperationKHR-operation-03436
    operation must be completed

Valid Usage (Implicit)
  • VUID-vkDestroyDeferredOperationKHR-device-parameter
    device must be a valid VkDevice handle

  • VUID-vkDestroyDeferredOperationKHR-operation-parameter
    If operation is not VK_NULL_HANDLE, operation must be a valid VkDeferredOperationKHR handle

  • VUID-vkDestroyDeferredOperationKHR-pAllocator-parameter
    If pAllocator is not NULL, pAllocator must be a valid pointer to a valid VkAllocationCallbacks structure

  • VUID-vkDestroyDeferredOperationKHR-operation-parent
    If operation is a valid handle, it must have been created, allocated, or retrieved from device

Host Synchronization
  • Host access to operation must be externally synchronized

To query the number of additional threads that can usefully be joined to a deferred operation, call:

// Provided by VK_KHR_deferred_host_operations
uint32_t vkGetDeferredOperationMaxConcurrencyKHR(
    VkDevice                                    device,
    VkDeferredOperationKHR                      operation);
  • device is the device which owns operation.

  • operation is the deferred operation to be queried.

The returned value is the maximum number of threads that can usefully execute a deferred operation concurrently, reported for the state of the deferred operation at the point this command is called. This value is intended to be used to better schedule work onto available threads. Applications can join any number of threads to the deferred operation and expect it to eventually complete, though excessive joins may return VK_THREAD_DONE_KHR immediately, performing no useful work.

If operation is complete, vkGetDeferredOperationMaxConcurrencyKHR returns zero.

If operation is currently joined to any threads, the value returned by this command may immediately be out of date.

If operation is pending, implementations must not return zero unless at least one thread is currently executing vkDeferredOperationJoinKHR on operation. If there are such threads, the implementation should return an estimate of the number of additional threads which it could profitably use.

Implementations may return 232-1 to indicate that the maximum concurrency is unknown and cannot be easily derived. Implementations may return values larger than the maximum concurrency available on the host CPU. In these situations, an application should clamp the return value rather than oversubscribing the machine.

The recommended usage pattern for applications is to query this value once, after deferral, and schedule no more than the specified number of threads to join the operation. Each time a joined thread receives VK_THREAD_IDLE_KHR, the application should schedule an additional join at some point in the future, but is not required to do so.

Valid Usage (Implicit)
  • VUID-vkGetDeferredOperationMaxConcurrencyKHR-device-parameter
    device must be a valid VkDevice handle

  • VUID-vkGetDeferredOperationMaxConcurrencyKHR-operation-parameter
    operation must be a valid VkDeferredOperationKHR handle

  • VUID-vkGetDeferredOperationMaxConcurrencyKHR-operation-parent
    operation must have been created, allocated, or retrieved from device

The vkGetDeferredOperationResultKHR function is defined as:

// Provided by VK_KHR_deferred_host_operations
VkResult vkGetDeferredOperationResultKHR(
    VkDevice                                    device,
    VkDeferredOperationKHR                      operation);
  • device is the device which owns operation.

  • operation is the operation whose deferred result is being queried.

If no command has been deferred on operation, vkGetDeferredOperationResultKHR returns VK_SUCCESS.

If the deferred operation is pending, vkGetDeferredOperationResultKHR returns VK_NOT_READY.

If the deferred operation is complete, it returns the appropriate return value from the original command. This value must be one of the VkResult values which could have been returned by the original command if the operation had not been deferred.

Valid Usage (Implicit)
  • VUID-vkGetDeferredOperationResultKHR-device-parameter
    device must be a valid VkDevice handle

  • VUID-vkGetDeferredOperationResultKHR-operation-parameter
    operation must be a valid VkDeferredOperationKHR handle

  • VUID-vkGetDeferredOperationResultKHR-operation-parent
    operation must have been created, allocated, or retrieved from device

Return Codes
Success
  • VK_SUCCESS

  • VK_NOT_READY

Failure

None