Making Tensors

make_tensor is a utility function for creating tensors. Where possible, using make_tensor is preferred over directly declaring a tensor_t since it allows the tensor type to change in the future without breaking. See Creating Tensors for a detailed walkthrough on creating tensors.

make_tensor provides numerous overloads for different arguments and use cases:

Return by Value

Added in version 0.1.0.

template<typename T, int RANK>
auto matx::make_tensor(const index_t (&shape)[RANK], matxMemorySpace_t space = MATX_MANAGED_MEMORY, cudaStream_t stream = 0)

Create a tensor with a C array for the shape using implicitly-allocated memory

Parameters:

• shape – Shape of tensor

• space – memory space to allocate in. Default is manged memory.

• stream – cuda stream to allocate in (only applicable to async allocations)

Returns:

New tensor

template<typename TensorType>
void matx::make_tensor(TensorType &tensor, const index_t (&shape)[TensorType::Rank()], matxMemorySpace_t space = MATX_MANAGED_MEMORY, cudaStream_t stream = 0)

Create a tensor with a C array for the shape using implicitly-allocated memory

Parameters:

• tensor – Tensor object to store newly-created tensor into

• shape – Shape of tensor

• space – memory space to allocate in. Default is manged memory.

• stream – cuda stream to allocate in (only applicable to async allocations)

template<typename T, typename ShapeType>
auto matx::make_tensor(ShapeType &&shape, matxMemorySpace_t space = MATX_MANAGED_MEMORY, cudaStream_t stream = 0)

Create a tensor from a conforming container type

Conforming containers have sequential iterators defined (both const and non-const). cuda::std::array and std::vector meet this criteria.

Parameters:

• shape – Shape of tensor

• space – memory space to allocate in. Default is managed memory.

• stream – cuda stream to allocate in (only applicable to async allocations)

Returns:

New tensor

template<typename TensorType>
void matx::make_tensor(TensorType &tensor, const std::vector<index_t> &shape, matxMemorySpace_t space, cudaStream_t stream)

Initialize a dynamic-rank tensor with owning storage and runtime shape.

This is the dynamic-rank equivalent of “placement” make_tensor.

template<typename TensorType>
auto matx::make_tensor(TensorType &tensor, matxMemorySpace_t space = MATX_MANAGED_MEMORY, cudaStream_t stream = 0)

Create a 0D tensor with implicitly-allocated memory.

Parameters:

• tensor – Tensor object to store newly-created tensor into

• space – memory space to allocate in. Default is managed memory memory.

• stream – cuda stream to allocate in (only applicable to async allocations)

Returns:

New tensor

template<typename T, int RANK>
auto matx::make_tensor(T *data, const index_t (&shape)[RANK], bool owning = false)

Create a tensor with user-defined memory and a C array

Parameters:

• data – Pointer to device data

• shape – Shape of tensor

• owning – If this class owns memory of data

Returns:

New tensor

template<typename TensorType>
auto matx::make_tensor(TensorType &tensor, typename TensorType::value_type *data, const index_t (&shape)[TensorType::Rank()])

Create a tensor with user-defined memory and a C array

Parameters:

• tensor – Tensor object to store newly-created tensor into

• data – Pointer to device data

• shape – Shape of tensor

Returns:

New tensor

template<typename T, typename ShapeType>
auto matx::make_tensor(T *data, ShapeType &&shape, bool owning = false)

Create a tensor with user-defined memory and conforming shape type

Parameters:

• data – Pointer to device data

• shape – Shape of tensor

• owning – If this class owns memory of data

Returns:

New tensor

template<typename TensorType>
auto matx::make_tensor(TensorType &tensor, typename TensorType::value_type *data, typename TensorType::shape_container &&shape)

Create a tensor with user-defined memory and conforming shape type

Parameters:

• tensor – Tensor object to store newly-created tensor into

• data – Pointer to device data

• shape – Shape of tensor

Returns:

New tensor

template<typename TensorType>
auto matx::make_tensor(TensorType &tensor, typename TensorType::value_type *ptr)

Create a 0D tensor with user-defined memory

Parameters:

• tensor – Tensor object to store newly-created tensor into

• ptr – Pointer to data

Returns:

New tensor

template<typename T, typename D>
auto matx::make_tensor(T *const data, D &&desc, bool owning = false)

Create a tensor with user-defined memory and an existing descriptor

Parameters:

• data – Pointer to device data

• desc – Tensor descriptor (tensor_desc_t)

• owning – If this class owns memory of data

Returns:

New tensor

template<typename TensorType>
auto matx::make_tensor(TensorType &tensor, typename TensorType::value_type *const data, typename TensorType::desc_type &&desc)

Create a tensor with user-defined memory and an existing descriptor

Parameters:

• tensor – Tensor object to store newly-created tensor into

• data – Pointer to device data

• desc – Tensor descriptor (tensor_desc_t)

Returns:

New tensor

template<typename T, typename D>
auto matx::make_tensor(D &&desc, matxMemorySpace_t space = MATX_MANAGED_MEMORY, cudaStream_t stream = 0)

Create a tensor with implicitly-allocated memory and an existing descriptor

Parameters:

• desc – Tensor descriptor (tensor_desc_t)

• space – memory space to allocate in. Default is managed memory memory.

• stream – cuda stream to allocate in (only applicable to async allocations)

Returns:

New tensor

template<typename TensorType>
auto matx::make_tensor(TensorType &&tensor, typename TensorType::desc_type &&desc, matxMemorySpace_t space = MATX_MANAGED_MEMORY, cudaStream_t stream = 0)

Create a tensor with implicitly-allocated memory and an existing descriptor

Parameters:

• tensor – Tensor object to store newly-created tensor into

• desc – Tensor descriptor (tensor_desc_t)

• space – memory space to allocate in. Default is managed memory memory.

• stream – cuda stream to allocate in (only applicable to async allocations)

Returns:

New tensor

template<typename T, int RANK>
auto matx::make_tensor(T *const data, const index_t (&shape)[RANK], const index_t (&strides)[RANK], bool owning = false)

Create a tensor with user-defined memory and C-array shapes and strides

Parameters:

• data – Pointer to device data

• shape – Shape of tensor

• strides – Strides of tensor

• owning – If this class owns memory of data

Returns:

New tensor

template<typename TensorType>
auto matx::make_tensor(TensorType &tensor, typename TensorType::value_type *const data, const index_t (&shape)[TensorType::Rank()], const index_t (&strides)[TensorType::Rank()])

Create a tensor with user-defined memory and C-array shapes and strides

Parameters:

• tensor – Tensor object to store newly-created tensor into

• data – Pointer to device data

• shape – Shape of tensor

• strides – Strides of tensor

Returns:

New tensor

Custom Allocator Support

template<typename T, int RANK, typename Allocator>
auto matx::make_tensor(const index_t (&shape)[RANK], Allocator &&alloc)

Create a tensor with custom allocator using C-array shape

Parameters:

• shape – Shape of tensor as C-array

• alloc – Custom allocator (PMR allocator, custom allocator pointer, etc.)

Returns:

New tensor

template<typename T, typename ShapeType, typename Allocator>
auto matx::make_tensor(ShapeType &&shape, Allocator &&alloc)

Create a tensor with custom allocator using conforming shape type

Parameters:

• shape – Shape of tensor (tuple, array, etc.)

• alloc – Custom allocator (PMR allocator, custom allocator pointer, etc.)

Returns:

New tensor

template<typename TensorType, typename Allocator>
void matx::make_tensor(TensorType &tensor, const index_t (&shape)[TensorType::Rank()], Allocator &&alloc)

Create a tensor with custom allocator using existing tensor reference

Parameters:

• tensor – Tensor object to store newly-created tensor into

• shape – Shape of tensor as C-array

• alloc – Custom allocator (PMR allocator, custom allocator pointer, etc.)

template<typename TensorType, typename ShapeType, typename Allocator>
void matx::make_tensor(TensorType &tensor, ShapeType &&shape, Allocator &&alloc)

Create a tensor with custom allocator using existing tensor reference and conforming shape

Parameters:

• tensor – Tensor object to store newly-created tensor into

• shape – Shape of tensor (tuple, array, etc.)

• alloc – Custom allocator (PMR allocator, custom allocator pointer, etc.)

mdspan Support

MatX can create non-owning tensors from cuda::std::mdspan and, when available, std::mdspan.

For C++20 and CUDA mdspan:

#include <cuda/std/mdspan>

int data[6]{};
using extents_type = cuda::std::extents<matx::index_t, 2, 3>;
cuda::std::mdspan<int, extents_type> span{data};

auto tensor = matx::make_tensor(span);

When C++23 std::mdspan is available, it can be used in the same way:

#include <mdspan>

int data[6]{};
using extents_type = std::extents<matx::index_t, 2, 3>;
std::mdspan<int, extents_type> span{data};

auto tensor = matx::make_tensor(span);

The tensor uses the mdspan’s original data, dimensions, and strides without copying the data. The original data must remain valid while the tensor is being used. Static and dynamic extents are supported with layout_right, layout_left, and layout_stride.

template<typename Mdspan>
auto matx::make_tensor(const Mdspan &span)

Takes an mdspan and creates a MatX tensor that looks at the same data. The returned tensor does not copy the underlying data but uses the same dimensions, strides, and data pointer as the mdspan. MatX will not delete the underlying data.

Parameters:

span – Tells us the data location, dimensions, and how it is arranged using strides

Returns:

A MatX tensor that uses the existing data

DLPack Support

Added in version 1.1.0.

template<typename TensorType>
auto matx::make_tensor(TensorType &tensor, DLManagedTensorVersioned *dlp_tensor)

Create a tensor from a versioned DLPack managed tensor and transfer ownership.

This consumes dlp_tensor, the deleter method will be called when the last MatX reference to the imported storage is released.

Parameters:

• tensor – Tensor object to store newly-created tensor into

• dlp_tensor – Pointer to a heap-allocated DLManagedTensorVersioned whose ownership is transferred to MatX

Added in version 1.1.0.

template<typename TensorType>
auto matx::make_tensor(TensorType &tensor, DLManagedTensor *dlp_tensor)

Create a tensor from a DLManagedTensor.

This consumes dlp_tensor, the deleter method will be called when the last MatX reference to the imported storage is released.

Parameters:

• tensor – Tensor object to store newly-created tensor into

• dlp_tensor – Pointer to a heap-allocated DLManagedTensor whose ownership is transferred to MatX

Return by Pointer

template<typename T, int RANK>
auto matx::make_tensor_p(const index_t (&shape)[RANK], matxMemorySpace_t space = MATX_MANAGED_MEMORY, cudaStream_t stream = 0)

Create a tensor with a C array for the shape using implicitly-allocated memory. Caller is responsible for deleting the tensor.

Parameters:

• shape – Shape of tensor

• space – memory space to allocate in. Default is managed memory.

• stream – cuda stream to allocate in (only applicable to async allocations)

Returns:

Pointer to new tensor

template<typename T, typename ShapeType>
auto matx::make_tensor_p(ShapeType &&shape, matxMemorySpace_t space = MATX_MANAGED_MEMORY, cudaStream_t stream = 0)

Create a tensor from a conforming container type

Conforming containers have sequential iterators defined (both const and non-const). cuda::std::array and std::vector meet this criteria. Caller is responsible for deleting tensor.

Parameters:

• shape – Shape of tensor

• space – memory space to allocate in. Default is managed memory memory.

• stream – cuda stream to allocate in (only applicable to async allocations)

Returns:

Pointer to new tensor

template<typename T, typename ShapeType>
auto matx::make_tensor_p(T *const data, ShapeType &&shape, bool owning = false)

Create a tensor with user-defined memory and conforming shape type

Parameters:

• data – Pointer to device data

• shape – Shape of tensor

• owning – If this class owns memory of data

Returns:

New tensor