Tensor Operations

Tensor Operations Overview

Vulkan Tensor Operations are operations performed by those SPIR-V Tensor Instructions which take an OpTypeTensorARM (representing a VkTensorViewARM). Read and write operations take tensor element coordinates as operands, and return elements from the tensor. Query operations return properties of the bound tensor.

Tensor Operations include the functionality of the following SPIR-V Tensor Instructions:

  • OpTensorReadARM reads one or more elements from the tensor.

  • OpTensorWriteARM writes one or more elements to the tensor.

  • OpTensorQuerySizeARM returns the size of the tensor descriptor that would be accessed. The tensor itself is not accessed.

Tensor Coordinate System

Each elements in a tensor can be addressed using a set of N integer coordinates where N is equal to the rank, i.e. number of dimensions, of the tensor.

Tensor Element Input Operations

Tensor element input instructions are SPIR-V tensor instructions that read from a tensor. Tensor element input operations are a set of steps that are performed on state, coordinates, and tensor element values while processing a tensor element input instruction, and which are common to some or all tensor element input instructions. They include the following steps, which are performed in the listed order:

For tensor element input instructions involving multiple tensor elements, these steps are applied for each tensor element that is used in the instruction.

Tensor Element Input Validation Operations

Tensor element input validation operations inspect instruction/tensor state or coordinates, and in certain circumstances cause the tensor element value to be replaced or become undefined. There are a series of validations that the tensor element undergoes.

Instruction/Tensor View Validation

There are a number of cases where a SPIR-V instruction can mismatch with the tensor view. In such cases the value of the tensor element returned is undefined.

These cases include:

  • The SPIR-V instruction’s tensor variable’s properties are not compatible with the tensor view:

    • The SPIR-V Tensor Element Type is not compatible with the tensor view’s format.

    • The SPIR-V Tensor Rank is not present.

    • The SPIR-V Tensor Rank is not the same as the value provided for VkTensorDescriptionARM::dimensionCount when creating the tensor used by the tensor view.

    • The SPIR-V Tensor Shape, if present, does not match the set of values provided for VkTensorDescriptionARM::pDimensions when creating the tensor used by the tensor view.

Tensor Element Coordinate Validation

Tensor element coordinates are validated against the size of the tensor.

If the tensor element coordinates do not satisfy all of the conditions

0 ≤ Coordinates[i] < Size[i] for 0 ≤ i < dimensionCount

where:

Size[i] is the size of the tensor along dimension i as provided via VkTensorDescriptionARM::pDimensions[i] when the tensor was created.

dimensionCount is the number of dimensions of the tensor as provided via VkTensorDescriptionARM::dimensionCount when the tensor was created.

, or the tensor descriptor is a null descriptor,

then the tensor element fails coordinate validation.

There are two cases to consider:

  1. Valid Tensor Element Coordinates

    • If the tensor element coordinates pass validation (that is, the coordinates lie within the tensor),

      then the tensor element value comes from the value in tensor memory.

  2. Out-of-Bounds Tensor Element

    • If the tensor element coordinates fail validation,

      then the tensor element is an invalid tensor element and tensor element replacement is performed.

Format Conversion

If required, tensor elements undergo a format conversion from the VkFormat of the tensor view to the element type of the tensor resource variable on which a read operation is performed.

Each element is converted based on its type and size (as defined in the Format Definition section for each VkFormat), using the appropriate equations in 16-Bit Floating-Point Numbers, Unsigned 11-Bit Floating-Point Numbers, Unsigned 10-Bit Floating-Point Numbers, Fixed-Point Data Conversion, and 8-bit booleans.

The compatibility between SPIR-V tensor element types and VkFormat is specified under Compatibility Between SPIR-V Tensor Element Types And Vulkan Formats.

Tensor Element Replacement

A tensor element is replaced if it is an invalid tensor element.

Invalid tensor elements are replaced as follows:

  • If the OpTensorReadARM instruction that caused an invalid tensor element to be accessed did not have a OutOfBoundsValueARM tensor operand present, then the value returned is the same as that produced by OpConstantNull for the type returned.

  • If the OpTensorReadARM instruction that caused an invalid tensor element to be accessed had a OutOfBoundsValueARM tensor operand present, then the value provided using OutOfBoundsValueARM is returned.

Tensor Element Output Operations

Tensor element output instructions are SPIR-V tensor instructions that write to a tensor. Tensor element output operations are a set of steps that are performed on state, coordinates, and tensor element values while processing a tensor element output instruction, and which are common to some or all tensor element output instructions. They include the following steps, which are performed in the listed order:

Tensor Element Output Validation Operations

Tensor element output validation operations inspect instruction/tensor state or coordinates, and in certain circumstances cause the write to have no effect. There are a series of validations that the tensor element undergoes.

Tensor Element Format Validation

If one of the following conditions is met, then an operation writing to the tensor causes the tensor’s memory to become undefined .

  • The SPIR-V instruction’s tensor variable’s properties are not compatible with the tensor view:

    • The SPIR-V Tensor Element Type is not compatible with the tensor view’s format.

    • The SPIR-V Tensor Rank is not present.

    • The SPIR-V Tensor Rank is not the same as the value provided for VkTensorDescriptionARM::dimensionCount when creating the tensor used by the tensor view.

    • The SPIR-V Tensor Shape, if present, does not match the set of values provided for VkTensorDescriptionARM::pDimensions when creating the tensor used by the tensor view.

Tensor Element Coordinate Validation

The tensor element coordinates are validated according to the same rules as for tensor element input coordinate validation.

If the tensor element fails coordinate validation, then the write has no effect.

Tensor Element Output Format Conversion

If required, tensor elements undergo a format conversion from the element type of the tensor resource variable on which a write operation is performed to the VkFormat of the tensor view.

Each element is converted based on its type and size (as defined in the Format Definition section for each VkFormat). Floating-point outputs are converted as described in Floating-Point Format Conversions and Fixed-Point Data Conversion. Boolean outputs are converted as described in 8-bit boolean. Integer outputs are converted such that their value is preserved.

The compatibility between SPIR-V tensor element types and VkFormat is specified under Compatibility Between SPIR-V Tensor Element Types And Vulkan Formats.

Tensor Query Instructions

OpTensorQuerySizeARM queries the size of a tensor whose descriptor would be accessed by a shader tensor operation.

It always returns 0 if the bound descriptor is a null descriptor.