Device Groups

Device groups are a way to have multiple physical devices (single-vendor) represented as a single logical device. If for example, an application have two of the same GPU, connected by some vendor-provided bridge interface, in a single system, one approach is to create two logical devices in Vulkan. The issue here is that there are limitations on what can be shared and synchronized between two VkDevice objects which is not a bad thing, but there are use cases where an application might want to combine the memory between two GPUs. Device Groups were designed for this use case by having an application create “sub-devices” to a single VkDevice. With device groups, objects like VkCommandBuffers and VkQueue are not tied to a single “sub-device” but instead, the driver will manage which physical device to run it on. Another usage of device groups is an alternative frame presenting system where every frame is displayed by a different “sub-device”.

There are two extensions, VK_KHR_device_group and VK_KHR_device_group_creation. The reason for two separate extensions is that extensions are either “instance level extensions” or “device level extensions”. Since device groups need to interact with instance level calls as well as device level calls, two extensions were created. There is also a matching SPV_KHR_device_group extension adding the DeviceGroup scope and a new DeviceIndex built-in type to shaders that allow shaders to control what to do for each logical device. If using GLSL there is also a GL_EXT_device_group extension that introduces a highp int gl_DeviceIndex; built-in variable for all shader types.