VK_EXT_swapchain_maintenance1
This proposal investigates and addresses a number of practical issues with the
VK_KHR_swapchain
specification.
1. Problem Statement
The following is a list of issues considered in this proposal:
-
It is not directly known when a semaphore used for presentation can be recycled.
-
Switching present modes requires a swapchain recreation
-
Upfront memory allocation is costly in startup time and memory
-
Unknown behavior when presenting a swapchain image to a surface with different dimensions
-
Impossible to release acquired images without presenting them
1.1. Recycling Present Semaphores
With VK_KHR_swapchain
, there is no way to provide feedback as to when a
semaphore used for presentation can be freed or recycled.
The application would have to infer this information from when the fence of the
next call to vkAcquireNextImageKHR
that returns the presented image’s index
has been signaled.
This is needlessly complicated.
1.2. Swapchain Recreation on Present Mode Change
Currently, when changing present modes, the Vulkan swapchain is required to be recreated. This is unnecessary if otherwise the swapchain is not changed, as the present mode does not affect the actual swapchain images and their associated memory. This is realistically affecting apps that switch between VK_PRESENT_MODE_SHARED_DEMAND_REFRESH_KHR and VK_PRESENT_MODE_SHARED_CONTINUOUS_REFRESH_KHR (in single-buffer applications). In that case, a flicker is noticed as the swapchain is recreated and the old contents of the swapchain is discarded.
1.3. Upfront Memory Allocation
Currently, Vulkan allows the application to record commands using any image
from the swapchain.
Additionally, the application may use VkBindImageMemorySwapchainInfoKHR
at
any time to bind an image to swapchain memory.
This requires swapchain implementations to allocate memory for all images
upfront.
This is costly both in startup time, and potentially memory if all images are
not eventually acquired.
1.4. Scaling Behavior
It is desirable for the application to specify a scaling behavior for when the
swapchain extents do not match the surface’s, such as during window resizing,
instead of receiving VK_ERROR_OUT_OF_DATE_KHR
.
See the proposal document for VK_EXT_surface_maintenance1
for details.
1.5. Releasing Acquired Images
When the swapchain needs to be recreated, it may be possible that an application has acquired images from the old swapchain that it is no longer interested in presenting to. However, it is unable to release those images before recreating the swapchain. As a result, the application is forced to present to an old swapchain that may no longer match the surface. Additionally, the memory allocated for the old and new swapchains coexist, increasing peak memory usage.
2. Solution Space
There are generally two possible approaches to resolving these problems:
-
Have the application directly interface with the window system, bypassing
VK_KHR_swapchain
. -
Improve upon
VK_KHR_swapchain
The benefits of the first approach are:
-
Total control of the swapchain implementation by the application, which would allow for prompt fixes of future issue
-
Works on existing drivers, removing any need for fallbacks
However, this approach results in duplicated effort among applications. A potential library with a swapchain implementation can be conceived which could be updated and shipped with applications independently from driver updates, though it lacks the benefit of driver updates bringing fixes and optimizations to all applications.
This proposal considers the second approach with the goal of improving the larger Vulkan ecosystem while leveraging existing swapchain implementations which most applications have come to rely on.
2.1. Recycling Present Semaphores
There are three potential solutions for this:
-
A status query for present operations, potentially identified through ids assigned with
VK_KHR_present_id
. -
A wait function similar to
vkWaitForPresentKHR
-
A fence passed to
vkQueuePresentKHR
that signals when the presentation engine no longer accesses the present semaphores
The last solution is adopted as it provides more power to the application (wait, in addition to query), as well as fitting better with event loops, other Vulkan APIs and future work to enable timeline semaphores.
2.2. Swapchain Recreation on Present Mode Change
Each present mode may require a different number of images.
Additionally, the VK_PRESENT_MODE_SHARED_DEMAND_REFRESH_KHR
and
VK_PRESENT_MODE_SHARED_CONTINUOUS_REFRESH_KHR
present modes may require a
different image memory layout compared with the rest of the present modes.
Switching between non-shared present modes may or may not be possible on some
implementations.
Regarding image count, potential solutions are:
-
Specify all potential present modes at swapchain creation time
-
The swapchain would then allocate as many images to satisfy all present modes
-
-
Increase the number of swapchain images as necessary when present mode is changed
-
This requires the application to use
vkGetSwapchainImagesKHR
after each change in present mode and adjust accordingly
-
Regarding the subsets of switchable present modes that the implementation supports, potential solutions are:
-
Specify a particular behavior for
VkSwapchainCreateInfoKHR::oldSwapchain
when the only change is in the present mode-
This is not straightforward and is error-prone
-
-
Allow modifying present modes as needed, such as during a
vkQueuePresentKHR
call-
Implementations may support switching between present modes only in disjoint subsets. This subsets can be queried. Alternatively, Vulkan could divide the present modes into shared and non-shared, with the latter conditional to a feature flag.
-
The adopted solution in this proposal is to allow applications to specify a
potential set of present modes at swapchain creation time.
A query must be used to determine whether these present modes are compatible
for dynamic switching.
Then, the application would pass the desired present mode to
vkQueuePresentKHR
.
With the solution to the "Upfront Memory Allocation" problem, the application can avoid paying any extra memory cost due to higher image counts until a present mode that uses that many images is used.
2.3. Upfront Memory Allocation
This can be explicitly opted in with a new swapchain create flag.
Using this flag would prohibit problematic scenarios, such as using
VkBindImageMemorySwapchainInfoKHR
with an image index that has never been
acquired, or recording command buffers using those swapchain images.
This amortizes the cost of memory allocation between the first few frames, using CPU time that may otherwise have gone idle waiting for the GPU. This will additionally resolve a number of memory issues:
-
If a present mode is specified at swapchain creation time which requires a larger number of images, but that is never actually used, the extra images would not have to consume memory.
-
When resizing the swapchain, peak memory increase is avoided by not actually allocating memory for the new swapchain. First memory allocation for the new swapchain could happen after some of the images from the old swapchain have been destroyed.
2.4. Scaling Behavior
The VK_EXT_surface_maintenance1
extension introduces scaling and gravity
behavior enums whose support can be queried from the surface.
At swapchain creation time, one of the supported behavior can be specified by
the application.
2.5. Releasing Acquired Images
There are a number of ways the applications could partially work around this issue, such as by deferring image acquisition, or recreating the swapchain only once all images from the old swapchain are presented.
However, in all implementations, releasing an acquired image without presenting it is no more complex than presenting it. The adopted solution is to expose this functionality in this extension.
3. Proposal
Introduced by this API are:
3.1. Feature
Advertising whether the implementation supports the functionality in this extension:
typedef struct VkPhysicalDeviceSwapchainMaintenance1FeaturesEXT {
VkStructureType sType;
void* pNext;
VkBool32 swapchainMaintenance1;
} VkPhysicalDeviceSwapchainMaintenance1FeaturesEXT;
3.2. Present Fence
To associate fences with the present operations for each swapchain, chain the
following to VkPresentInfoKHR
:
typedef struct VkSwapchainPresentFenceInfoEXT {
VkStructureType sType;
void* pNext;
uint32_t swapchainCount;
VkFence* pFences;
} VkSwapchainPresentFenceInfoEXT;
With swapchainCount
matching
VkSwapchainPresentFenceInfoEXT::swapchainCount
, each swapchain being
presented to will signal the fence once the application is allowed to destroy
or recycle the semaphores passed to vkPresentInfoKHR::pWaitSemaphores
.
3.3. Switching Present Modes
During creation of the swapchain, all potential present modes are specified by
chaining the following to VkSwapchainCreateInfoKHR
:
typedef struct VkSwapchainPresentModesCreateInfoEXT {
VkStructureType sType;
void* pNext;
uint32_t presentModeCount;
VkPresentModeKHR* pPresentModes;
} VkSwapchainPresentModesCreateInfoEXT;
The present modes given in pPresentModes
must be compatible for mode
switching.
This can be queried by use of VkSurfacePresentModeCompatibilityEXT
from the
VK_EXT_surface_maintenance1
extension.
The present mode can be changed by chaining the following to
VkPresentInfoKHR
:
typedef struct VkSwapchainPresentModeInfoEXT {
VkStructureType sType;
void* pNext;
uint32_t swapchainCount;
VkPresentModeKHR* pPresentModes;
} VkSwapchainPresentModeInfoEXT;
Where the elements of pPresentModes
can take any present mode specified in
VkSwapchainPresentModesCreateInfoEXT
during the creation of the respective
swapchain.
If not specified, the swapchain will continue to operate according to the last
specified present mode.
3.4. Swapchain Memory Allocation
To allow the swapchain to defer memory allocation for each image until it is
acquired through vkAcquireNextImageKHR
, specify the
VK_SWAPCHAIN_CREATE_DEFERRED_MEMORY_ALLOCATION_BIT_EXT
flag in
VkSwapchainCreateInfoKHR::flags
.
In that case, the application may still use vkGetSwapchainImagesKHR
to
retrieve the image handles, but it may not use any image whose index has never
been returned by vkAcquireNextImageKHR
; this includes recording commands
using such an image, or binding another image to its memory through
VkBindImageMemorySwapchainInfoKHR
.
As memory allocation for each image is deferred, so should the application defer the above operations for each image until it is first acquired.
3.5. Scaling Behavior
To specify the scaling behavior of the swapchain, chain the following to
VkSwapchainCreateInfoKHR
:
typedef struct VkSwapchainPresentScalingCreateInfoEXT {
VkStructureType sType;
void* pNext;
VkPresentScalingFlagsEXT scalingBehavior;
VkPresentGravityFlagsEXT presentGravityX;
VkPresentGravityFlagsEXT presentGravityY;
} VkSwapchainPresentScalingCreateInfoEXT;
The values specified in scalingBehavior
, presentGravityX
and
presentGravityY
must be supported by the surface.
This can be queried by use of VkSurfacePresentScalingCapabilitiesEXT
from the
VK_EXT_surface_maintenance1
extension.
3.6. Releasing Acquired Images
To release previously acquired images back to the swapchain, call
vkReleaseSwapchainImagesEXT
:
VKAPI_ATTR VkResult VKAPI_CALL vkReleaseSwapchainImagesEXT(
VkDevice device,
const VkReleaseSwapchainImagesInfoEXT* pReleaseInfo);
VkReleaseSwapchainImagesInfoEXT
is defined as:
typedef struct VkReleaseSwapchainImagesInfoEXT {
VkStructureType sType;
const void* pNext;
VkSwapchainKHR swapchain;
deUint32 imageIndexCount;
const deUint32* pImageIndices;
} VkReleaseSwapchainImagesInfoEXT;