coopVecMatMulAddPacked

Description

Multiply a matrix with a cooperative vector and add a bias vector to the result. Given a M-row by K-col matrix, a K-element column vector input, and a M-element vector bias, computes matrix*input+bias, and returns a M-element vector.

Signature

/// Requires Capability Set 1:
CoopVec<T, M> coopVecMatMulAddPacked<T, int M, int PackedK, U>(
    CoopVec<U, PackedK> input,
    CoopVecComponentType inputInterpretation,
    int k,
    RWByteAddressBuffer matrix,
    int matrixOffset,
    CoopVecComponentType matrixInterpretation,
    RWByteAddressBuffer bias,
    int biasOffset,
    CoopVecComponentType biasInterpretation,
    CoopVecMatrixLayout memoryLayout,
    bool transpose,
    uint matrixStride)
    where T : __BuiltinArithmeticType
    where U : __BuiltinArithmeticType;

/// Requires Capability Set 1:
CoopVec<T, M> coopVecMatMulAddPacked<T, int M, int PackedK, U>(
    CoopVec<U, PackedK> input,
    CoopVecComponentType inputInterpretation,
    int k,
    ByteAddressBuffer matrix,
    int matrixOffset,
    CoopVecComponentType matrixInterpretation,
    ByteAddressBuffer bias,
    int biasOffset,
    CoopVecComponentType biasInterpretation,
    CoopVecMatrixLayout memoryLayout,
    bool transpose,
    uint matrixStride)
    where T : __BuiltinArithmeticType
    where U : __BuiltinArithmeticType;

/// Requires Capability Set 2:
CoopVec<T, M> coopVecMatMulAddPacked<T, int M, int PackedK, U, IgnoredBufferElementType>(
    CoopVec<U, PackedK> input,
    CoopVecComponentType inputInterpretation,
    int k,
    RWStructuredBuffer<IgnoredBufferElementType, DefaultDataLayout> matrix,
    int matrixOffset,
    CoopVecComponentType matrixInterpretation,
    RWStructuredBuffer<IgnoredBufferElementType, DefaultDataLayout> bias,
    int biasOffset,
    CoopVecComponentType biasInterpretation,
    CoopVecMatrixLayout memoryLayout,
    bool transpose,
    uint matrixStride)
    where T : __BuiltinArithmeticType
    where U : __BuiltinArithmeticType;

/// Requires Capability Set 2:
CoopVec<T, M> coopVecMatMulAddPacked<T, int M, int PackedK, U, IgnoredBufferElementType>(
    CoopVec<U, PackedK> input,
    CoopVecComponentType inputInterpretation,
    int k,
    StructuredBuffer<IgnoredBufferElementType, DefaultDataLayout> matrix,
    int matrixOffset,
    CoopVecComponentType matrixInterpretation,
    StructuredBuffer<IgnoredBufferElementType, DefaultDataLayout> bias,
    int biasOffset,
    CoopVecComponentType biasInterpretation,
    CoopVecMatrixLayout memoryLayout,
    bool transpose,
    uint matrixStride)
    where T : __BuiltinArithmeticType
    where U : __BuiltinArithmeticType;

/// Requires Capability Set 3:
CoopVec<T, M> coopVecMatMulAddPacked<T, int M, int PackedK, U>(
    CoopVec<U, PackedK> input,
    CoopVecComponentType inputInterpretation,
    int k,
    Ptr<void, Access.ReadWrite, AddressSpace.Device> matrixPtr,
    CoopVecComponentType matrixInterpretation,
    Ptr<void, Access.ReadWrite, AddressSpace.Device> biasPtr,
    CoopVecComponentType biasInterpretation,
    CoopVecMatrixLayout memoryLayout,
    bool transpose,
    uint matrixStride)
    where T : __BuiltinArithmeticType
    where U : __BuiltinArithmeticType;

Generic Parameters

T: __BuiltinArithmeticType

M : int

PackedK : int

U: __BuiltinArithmeticType

IgnoredBufferElementType

Parameters

input : CoopVec<U, PackedK>

The input cooperative vector to multiply with the matrix.

inputInterpretation : CoopVecComponentType

Specifies how to interpret the values in the input vector (e.g. as packed values).

k : int

The number of columns in the matrix.

matrix : RWByteAddressBuffer

The matrix buffer to multiply with.

matrixOffset : int

Byte offset into the matrix buffer.

matrixInterpretation : CoopVecComponentType

Specifies how to interpret the values in the matrix.

bias : RWByteAddressBuffer

The bias buffer to add after multiplication.

biasOffset : int

Byte offset into the bias buffer.

biasInterpretation : CoopVecComponentType

Specifies how to interpret the values in the bias vector.

memoryLayout : CoopVecMatrixLayout

Specifies the memory layout of the matrix (row-major or column-major).

transpose : bool

Whether to transpose the matrix before multiplication.

matrixStride : uint

The stride between matrix rows/columns in bytes.

matrix : ByteAddressBuffer

The matrix buffer to multiply with.

bias : ByteAddressBuffer

The bias buffer to add after multiplication.

matrix : RWStructuredBuffer<IgnoredBufferElementType, DefaultDataLayout>

The matrix buffer to multiply with.

bias : RWStructuredBuffer<IgnoredBufferElementType, DefaultDataLayout>

The bias buffer to add after multiplication.

matrix : StructuredBuffer<IgnoredBufferElementType, DefaultDataLayout>

The matrix buffer to multiply with.

bias : StructuredBuffer<IgnoredBufferElementType, DefaultDataLayout>

The bias buffer to add after multiplication.

matrixPtr : Ptr<void, Access.ReadWrite, AddressSpace.Device>

biasPtr : Ptr<void, Access.ReadWrite, AddressSpace.Device>

Return value

A new cooperative vector containing the result of the matrix multiplication with added bias.

Remarks

Unlike coopVecMatMulAdd, this function supports packed input interpretations where multiple values can be packed into each element of the input vector. The k parameter specifies the actual number of values to use from the packed input.

Depending on target hardware, some combinations of inputInterpretation, matrixInterpretation and memoryLayout may not be supported. For example, CoopVecComponentType.Float32 is not widely supported. Developers should query device properties through the host graphics API to find out which interpretations are supported.

Transposing is not supported when memoryLayout is RowMajor or ColumnMajor, and transpose must be false. Not all component types support transposing. When memoryLayout is InferencingOptimal or TrainingOptimal, matrixStride is ignored.

Availability and Requirements

Capability Set 1

Defined for the following targets:

hlsl

Available in all stages.

glsl

Available in all stages.

cpp

Available in all stages.

cuda

Available in all stages.

Requires capability: optix_coopvec.

spirv

Available in all stages.

Requires capability: spvCooperativeVectorNV.

Capability Set 2

Defined for the following targets:

spirv

Available in all stages.

Requires capability: spvCooperativeVectorNV.

cuda

Available in all stages.

Requires capability: optix_coopvec.

Capability Set 3

Defined for the following targets:

spirv

Available in all stages.

Requires capability: spvCooperativeVectorNV.